If we have the LLVM port installed, CMake might pick up some of the
tools installed as part of it (`llvm-ar`, `llvm-strip`, etc.) instead of
the ones belonging to the host toolchain. These, of course, can't be run
on the host platform, so builds would eventually fail. This made it
impossible to rebuild the LLVM toolchain.
We now set these variables explicitly when compiling the LLVM runtime
libraries in order to avoid this issue.
This will come in handy if we want to use the LLVM port with a GNU host
compiler.
As of version 13, libc++ uses `__attribute__((using_if_exists))` to
import global LibC functions into the `std` namespace, which allows some
symbols to be absent. GCC does not support this attribute, so it fails
to build libc++ due to some obscure `wchar.h` functions. This means that
cross-compiling libc++ is not possible; and on-target builds would be
tedious, so we'll be better off using the toolchain's `libstdc++`.
The goal of these more recent additions to the Dockerfile is to provide
a working copy of SerenityOS with the toolchain prebuilt. To me, these
additions feel misplaced:
- The toolchain is built assuming the i686 architecture, which may not
be what you want.
- You get a shallow clone of the project limiting you in your abilities
to navigate through the project's history or bisect.
- There's this awkward directory structure of `/serenity/serenity-git`
and `/serenity/out`.
The Dockerfile is immensely useful for building SerenityOS in a
containerized environment, separate from the host's environment. If we
want to automate builds, we can always use CI or extend this image to
do so. For now, let's remove the `git clone` and associated actions.
Fixes#9310.
If we want to use clang-tidy on the codebase, we'll need to build
clang-tidy from an LLVM that has been patched and built with Serenity
cross-compilation support.
Serenity defines a protected range of memory that must not be mmapped,
and is apparently reserved for kernel tasks. In this case, the protected
range is anything below 0x800000.
However, in its default setting, binutils chooses the memory address
0x400000 as the mapping address for executables that do not have PIE
enabled, resulting in mmap being unable to map the file unless the load
address has been overwritten at link time or if it's a PIE.
To mitigate this, move the default base address somewhere outside of
that range (and preferably not anywhere close near the beginning of the
useable virtual memory space, to avoid running into it during sequential
allocations).
We were previously using TRY_COMPILE_TARGET_TYPE to bypass the compiler
check at the beginning of the CMake build, since we don't have LibC
available and therefore can't link at that point.
However, this breaks a lot of assumptions in try_compile when it comes
to library checks. While this was the main idea behind our usage of the
flag, it also has some really nasty side effects when software wants
to find out what library a symbol is in.
Instead, just manually tell CMake that our compiler works as intended
and keep the target type setting at its default.
`CMAKE_INSTALL_PREFIX` is supposed to be the in-system installation
path. The sysroot path on the host doesn't belong there, since other
applications will duplicate that path when applying their respective
sysroot.
This commit updates the Clang toolchain's version to 13.0.0, which comes
with better C++20 support and improved handling of new features by
clang-format. Due to the newly enabled `-Bsymbolic-functions` flag, our
Clang binaries will only be 2-4% slower than if we dynamically linked
them, but we save hundreds of megabytes of disk space.
The `BuildClang.sh` script has been reworked to build the entire
toolchain in just three steps: one for the compiler, one for GNU
binutils, and one for the runtime libraries. This reduces the complexity
of the build script, and will allow us to modify the CI configuration to
only rebuild the libraries when our libc headers change.
Most of the compile flags have been moved out to a separate CMake cache
file, similarly to how the Android and Fuchsia toolchains are
implemented within the LLVM repo. This provides a nicer interface than
the heaps of command-line arguments.
We no longer build separate toolchains for each architecture, as the
same Clang binary can compile code for multiple targets.
The horrible mess that `SERENITY_CLANG_ARCH` was, has been removed in
this commit. Clang happily accepts an `i686-pc-serenity` target triple,
which matches what our GCC toolchain accepts.
This allows the linker to link against these dynamic libraries when
compiling libc++/libunwind, without having to do a separate
bootstrapping LibC build.
Without this change, libc++ would fail to pick up the need to link to
`LibPthread` if no prior builds of it existed. Because of this, we'd
immediately have an assertion failure in SystemServer, as mutexes are
used for the safe construction of function-local static variables.
I used "git grep -FIn http://" to find all occurrences, and looked at
each one. If an occurrence was really just a link, and if a https
version exists, and if our Browser can access it at least as well as the
http version, then I changed the occurrence to https.
I'm happy to report that I didn't run into a single site where Browser
can't deal with the https version.
Replace the old logic where we would start with a host build, and swap
all the CMake compiler and target variables underneath it to trick
CMake into building for Serenity after we configured and built the Lagom
code generators.
The SuperBuild creates two ExternalProjects, one for Lagom and one for
Serenity. The Serenity project depends on the install stage for the
Lagom build. The SuperBuild also generates a CMakeToolchain file for the
Serenity build to use that replaces the old toolchain file that was only
used for Ports.
To ensure that code generators are rebuilt when core libraries such as
AK and LibCore are modified, developers will need to direct their manual
`ninja` invocations to the SuperBuild's binary directory instead of the
Serenity binary directory.
This commit includes warning coalescing and option style cleanup for the
affected CMakeLists in the Kernel, top level, and runtime support
libraries. A large part of the cleanup is replacing USE_CLANG_TOOLCHAIN
with the proper CMAKE_CXX_COMPILER_ID variable, which will no longer be
confused by a host clang compiler.
Ninja disables its fancy output mode when it's not writing to a TTY.
So don't pipe its output into something else, so that it writes to
a TTY if the invoking terminal is a TTY.
`LLVM_LLVM_BUILD_LLVM_DYLIB` does not exist, so passing this does
nothing but make CMake warn.
However, since we pass `LLVM_LINK_LLVM_DYLIB`, `LLVM_BUILD_LLVM_DYLIB`
(the correct spelling) defaults to true anyways. So let's pass fewer
flags.
No behavior change, but fixes a CMake warning.
I locally modified Meta/serenity.sh to pass `--dev` to BuildIt.sh
in build_toolchain(). Then I ran `Meta/serenity.sh rebuild-toolchain`,
cd'd into Toolchain/Tarballs/binutils-2.37, `git add`ed unadded files in
`git status`, and then ran `git diff > ../../Patches/binutils.patch`.
Then I did the same for Toolchain/Tarballs/gcc-11.2.0 (and was careful
not to `git add` serenity-kernel.h, since that's created by
Toolchain/BuildIt.sh).
No behavior change. This just rewrites the patch like git writes it.
This library is used by virtually all executables in the Clang
toolchain. By default, it is linked statically, which leads to huge
file sizes and us running out of artifact storage disk space on CI.
This contains all the bits and pieces necessary to build a Clang binary
that will correctly compile SerenityOS.
I had some trouble with getting LLVM building with a single command, so
for now, I decided to build each LLVM component in a separate command
invocation. In the future, we can also make the main llvm build step
architecture-independent, but that would come with extra work to make
library and include paths work.
The binutils build invocation and related boilerplate is duplicated
because we only use `objdump` from GNU binutils in the Clang toolchain,
so most features can be disabled.
CMake specifies -arch arm64 for our toolchain. Unfortunately that's an
option GCC only understands when built for macOS. This causes the build
to fail.
I haven't been able to get CMake to not specify that option so this adds
a dummy option to GCC.
Previously we'd place the QEMU binaries into the architecture-specific
toolchain directory. This is a problem because the BuildIt.sh script
clears those directories which also removes the QEMU binaries users
may have built earlier. Also, the QEMU binaries are not specific to
the target architecture.
Docker is a nice way of doing build automation, or just
containerizing builds for increased safety and isolating unstable
packages. The old Dockerfile in the toolchain did not satisfy these
needs. The new Dockerfile is known to run successfully on Docker
version 20.10.7. It clones the SerenityOS repo and builds the
toolchain. In this way, it is intended to be a starting point for other
Docker images that can e.g. run builds. For example, one can simply run
this docker image as-is, exec a shell in it and run a build there.
