Override the vcpkg/scripts/detect_compiler behavior of always pulling
$CC and $CXX at the time that vcpkg install is determined to need called
by forcing $ENV{CXX} and $ENV{CC} to our CMake-determined compiler.
This prevents strange behavior such as running the following:
./Meta/ladybird.sh run
make changes...
ninja -C Build/ladybird
Where the second build step would be run without CC or CXX set in the
environment, causing a total cache miss from vcpkg and a full rebuild.
It also helps prevent full rebuilds when an IDE passes a slightly
different compiler to the build step than ladybird.sh.
This ensures that vcpkg downloads and builds all dependencies for
Android. We add it as a CMAKE_TOOLCHAIN_FILE that then chainloads the
Android NDK's toolchain file, as per the vcpkg documentation.
Also add a vcpkg command to ladybird.sh to ensure that vcpkg is setup,
and use a local binary cache for vcpkg build and install media to
avoid cluttering $XDG_CACHE_HOME.
We will use sqlite3 as a replacement for LibSQL. Using a tried-and-true
database will allow us to avoid maintaining our an incomplete, non-ACID,
and less performant implementation. It also means we do not have to
launch and manage the singleton SQLServer process.
Apart from bumping the toolchain Clang's and port's version, this commit
completely overhauls the way LLVM toolchain is built.
First, it gets rid of a complicated two-stage process of first compiling
clang and compiler-rt builtins and then building libunwind, libc++abi,
and libc++ -- it is possible to create a complete cross-compilation
toolchain in a single CMake invocation with a modern LLVM. Moreover, the
old method was inherently unsupported and subtly broken.
Next, it utilizes full potential of the Stubs "framework". Now we are
even able to compile Clang with -Wl,-z,defs which makes one of the
patches obsolete and the whole installation less error-prone. Note that
it comes at a cost of complicating the bootstrap process on a completely
novel architecture but this hopefully won't happen often.
Lastly, it fixes handling of the -no*lib* family of flags in the
Serenity LLVM driver and correctly uses -nostartfiles in conjunction
with stubs to make necessary CMake configure-time checks succeed.
A Python script is much easier to maintain than the organically grown
variable mess that was run.sh.
For now, the script inherits most environment variable modifiability
from the shell script, but this is not a requirement.
While porting this script, a couple of improvements have been made:
- Spaces (especially in paths) cannot break most arguments anymore.
Exceptions are environment variables specifying multiple arguments on
purpose, these should be replaced in the future anyways.
- Force control over virtualization is now possible with
SERENITY_VIRTUALIZATION_SUPPORT. If set to 0, this variable was
sometimes ignored before.
- Handling Windows native QEMU is much more robust. Multiple incorrect
checks for WSL, but not Windows native QEMU, were used before. This
would also allow disabling native Windows QEMU much more easily in the
future, which is necessary for GDB.
- Various machine types had wrong or outdated arguments, such as qn.
Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
This makes CMake pass `-fpie` instead of `-fpic` to the compiler when
building the Kernel and userland *executables*. This allows the compiler
to make certain optimizations based on the fact that the code will be
used in an executable, such as not having to emit `.localalias` symbols.
This leads to a 450 KiB decrease in the size of the Kernel binary.
This is a minor bugfix release, which to my knowledge contains nothing
of importance to us. However, there is one QoL change to our patches.
We no longer force `-fpic` in the compiler driver, and instead use the
`--enable-default-pie` configure option to generate position-independent
code suitable for executables. For building shared libraries, the
`-fpic` flag must be specified explicitly.
While LLD and mold support RELR "packed" relocations on all
architectures, the BFD linker currently only implements them on x86-64
and POWER.
This fixes two issues:
- The Kernel had it enabled even for AArch64 + GCC, which led to the
following being printed: `warning: -z pack-relative-relocs ignored`.
- The userland always had it disabled, even in the supported AArch64 +
Clang/mold scenarios.
Since https://reviews.llvm.org/D131441, libc++ must be included before
LibC. As clang includes libc++ as one of the system includes, LibC
must be included after those, and the only correct way to do that is
to install LibC's headers into the sysroot.
Targets that don't link with LibC yet require its headers for one
reason or another must add install_libc_headers as a dependency to
ensure that the correct headers have been (re)installed into the
sysroot.
LibC/stddef.h has been dropped since the built-in stddef.h receives
a higher include priority.
In addition, string.h and wchar.h must
define __CORRECT_ISO_CPP_STRING_H_PROTO and
_LIBCPP_WCHAR_H_HAS_CONST_OVERLOADS respectively in order to tell
libc++ to not try to define methods implemented by LibC.
This is a mostly straight-forward rebase of our patches on top of
13.1.0. The spec files needed a change, as GCC no longer supports STABS
debug information, but we were building GCC with support for it.
Highlights of this release include static `operator()`, The Equality
Operator You Are Looking For and extended `constexpr` support.
The install() command used by 1e36d54493
installs the provided file into the *directory* named by the DESTINATION
parameter. So if we ask it to install pci.ids to /res/pci.ids, the final
destination will be /res/pci.ids/pci.ids.
`file(COPY_FILE ...)` fails if the target directory does not already
exist, whereas `install(FILES ... DESTINATION ...)` happily creates all
the required directories.
This ensures we only need to download these files once for all build
configurations. We similarly download the UCD, CLDR, and TZDB to this
cache directory as well.
"image" was an alias for "qemu-image".
I want to add an `image` userland utility, which clashes with that
shortname.
So remove the existing "image" target. It was just an alias for
"qemu-image".
If you use serenity.sh to build, nothing changes. This only affects you
if you run ninja manually -- you now have to say `ninja qemu-image` to
build the disk image.
Clean up the Wasm spec tests CMake rules to extract and compile the wat
files into wasm files in the LibWasm binary directory instead of its
source directory. Also make the rules more robust to missing host tools,
and use more CMake install rules for the test files rather than relying
on build-root-filesystem.sh. Add some FIXMEs for later, we really
shouldn't be doing installation of test files into /home/anon at the
build-root-filesystem stage in $CURRENT_YEAR. Tests go in /usr/Tests
Tell CMake to not create a new policy scope for the
(lagom|serenity|common)_options.cmake helpers, and lets us set common
policies for both projects in common_options.cmake that actually apply
to the rest of the project, instead of just common_options.cmake itself.
Problem: cmake cannot handle gzip files (see
https://gitlab.kitware.com/cmake/cmake/-/issues/23054 for more
details).
Instead of downloading the compressed (*.gz) USB and PCI ids,
we not download the raw uncompressed files. The sizes we "loose"
due to downloading such files are meaningless.
This are the file sizes:
```
diego@debian:~/$ ls -lh pci.ids{,.gz} usb.ids{,.gz}
-rw-r--r-- 1 diego diego 1.3M Aug 7 04:15 pci.ids
-rw-r--r-- 1 diego diego 300K Aug 7 04:15 pci.ids.gz
-rw-r--r-- 1 diego diego 700K May 20 22:34 usb.ids
-rw-r--r-- 1 diego diego 245K May 20 22:34 usb.ids.gz
```
Newer cmake's have internal functions to un-compress files. These
functions will work on pure windows - as well as linux. This
eliminates the need to search for external tools (TAR,GZIP,ZIP) - and
helps fixing #9866.
In order to finally fix#9866 we need to decide to bump the cmake
version requirements and remove the checks. If we demand a newer cmake
version, we will loose Ubuntu 20.04 as a build target - as it ships
with CMake 3.16.
For now - we keep compatibility with CMake 3.16 - and only if CMake
3.18 as been found - we use its new functionality.
Remove the Corrosion dependency, and use the now-builtin
add_jakt_executable function from the Jakt install rules to build our
example application.
By using find_package(Jakt), we now have to set ENABLE_JAKT manually on
both serenity and Lagom at the same time, so the preferred method to do
this for now is:
cmake -B Build/superbuild<arch><toolchain> \
-S Meta/CMake/Superbuild \
-DENABLE_JAKT=ON \
-DJAKT_SOURCE_DIR=/path/to/jakt
Where omitting JAKT_SOURCE_DIR will still pull from the main branch of
SerenityOS/jakt. This can be done after runing Meta/serenity.sh run.
The FLAC "spec tests", or rather the test suite by xiph that exercises
weird FLAC features and edge cases, can be found at
https://github.com/ietf-wg-cellar/flac-test-files and is a good
challenge for our FLAC decoder to become more spec compliant. Running
these tests is similar to LibWasm spec tests, you need to pass
INCLUDE_FLAC_SPEC_TESTS to CMake.
As of integrating these tests, 23 out of 63 fail. :yakplus:
The current emoji_txt.cmake does not handle download errors (which were
a common source of issues in the build problems channel) or Unicode
versioning. These are both handled by unicode_data.cmake. Move the
download to unicode_data.cmake so that we can more easily handle next
month's Unicode 15 release.
Instead of manually updating emoji.txt whenever new emoji are added,
we use Unicode's emoji-test.txt to generate emoji.txt on each build,
including only the emojis that Serenity supports at that time.
By using emoji-test.txt, we can also include all forms of each emoji
(fully-qualified, minimally-qualified, and unqualified) which can be
helpful when double-checking how certain forms are handled.
Not sure why that happens and how it worked until now, but we need to be
more precise about the location of PCI and USB IDs when decompressing
them while building the OS.
In commit 02e8f29560 we started exporting
the `CMAKE_INSTALL_*DIR` variables without prefix in order to make
better use of the actual `PREFIX` settings.
However, commands like `file(MAKE_DIRECTORY ...)` don't understand the
GNUInstallDirs way of building paths, so we ended up creating
directories in our main project directory by accident.
Fix that by manually adding the correct prefix onto the path.
In preparation for future refactoring of Lagom, let's use the variables
from GNUInstallDirs as much as possible for the helper macros and other
scripts used by the main build already.
