I've been wanting to do this for a long time. It's time we start being
consistent about how this stuff works.
The new convention is:
- "LibFoo" is a userspace library that provides the "Foo" namespace.
That's it :^) This was pretty tedious to convert and I didn't even
start on LibGUI yet. But it's coming up next.
Unparented GActions are still parented to the application like before,
making them globally available.
This makes it possible to have actions that work whenever a specific
window is active, no matter which widget is currently focused. :^)
This is a complete reimplementation of CArgsParser with a different API.
Now, CArgsParser properly supports and distinguishes between:
* Positional arguments (required or not)
* Options
Options can be short and/or long.
The API allows you to add custom option and argument types. A few types are
pre-implemented for convenience:
* Boolean options (take no value)
* String and integer options (take a required value)
* String and integer arguments
* Vector-of-string arguments
This commit doesn't include changes for all the users of CArgsParser (see next
commit for that).
The "stay_within" parameter to CObject::dispatch_event() optionally
specifies a node in the CObject parent chain where event dispatch
should stop bubbling upwards.
Since event dispatch is done recursively, this was not working right,
as we would simply return from the innermost dispatch loop, leaving
the event un-accepted, which meant that the penultimately inner
dispatch loop would pick up the event and keep bubbling it anyway.
This made it possible for events to jump across window boundaries
within an application, in cases where one window was a CObject ancestor
of another window. This is typically the case with dialog windows.
Fix#1078.
A process has one of three veil states:
- None: unveil() has never been called.
- Dropped: unveil() has been called, and can be called again.
- Locked: unveil() has been called, and cannot be called again.
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.
For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.
Going forward, all new source files should include a license header.
It used to only read the data it could get without blocking. Andreas says this
was intentional, but it's counterintuitive and no code that uses read_all()
actually expects it to return only a part of the data. So change it to always
read data until an EOF (or an error) is received.
CArgsParser::parse_next_param did not properly ensure that, when
a param required a following argument, there were enough parameters left to
complete the parse. This meant that params_left could become negative,
avoiding parse_next_param's termination condition, and cause a segfault
when reading from argv with an out of bounds index.
This fixes the check to ensure that we do in fact have the right amount
of parameters and also adds an assertion to ensure that params_left does
not become negative.
Threads now have numeric priorities with a base priority in the 1-99
range.
Whenever a runnable thread is *not* scheduled, its effective priority
is incremented by 1. This is tracked in Thread::m_extra_priority.
The effective priority of a thread is m_priority + m_extra_priority.
When a runnable thread *is* scheduled, its m_extra_priority is reset to
zero and the effective priority returns to base.
This means that lower-priority threads will always eventually get
scheduled to run, once its effective priority becomes high enough to
exceed the base priority of threads "above" it.
The previous values for ThreadPriority (Low, Normal and High) are now
replaced as follows:
Low -> 10
Normal -> 30
High -> 50
In other words, it will take 20 ticks for a "Low" priority thread to
get to "Normal" effective priority, and another 20 to reach "High".
This is not perfect, and I've used some quite naive data structures,
but I think the mechanism will allow us to build various new and
interesting optimizations, and we can figure out better data structures
later on. :^)
This removes a bunch of JsonValue copying from the hot path in thread
statistics fetching.
Also pre-size the thread statistics vector since we know the final size
up front. :^)
LibCore timers now have a TimerShouldFireWhenNotVisible flag which is
set to "No" by default.
If "No", the timer will not be fired by the event loop if it's within
a CObject tree whose nearest GWindow ancestor is currently not visible
for timer purposes. (Specificially, this means that the window is
either minimized or fully occluded, and so does not want to fire timers
just to update the UI.)
This is another nice step towards a calm and serene operating system.
This is memory that's loaded from an inode (file) but not modified in
memory, so still identical to what's on disk. This kind of memory can
be freed and reloaded transparently from disk if needed.
Dirty private memory is all memory in non-inode-backed mappings that's
process-private, meaning it's not shared with any other process.
This patch exposes that number via SystemMonitor, giving us an idea of
how much memory each process is responsible for all on its own.
Compiling LibCore on macOS is needed if one wants to compile host tools (like IPCCompiler) on a non Linux host. These changes could be possibly reverted once "event loop" functionality and "base library" (Vector, String etc.) will be split in two separate libraries,
updating all relevant projects.
Color themes are loaded from .ini files in /res/themes/
The theme can be switched from the "Themes" section in the system menu.
The basic mechanism is that WindowServer broadcasts a SharedBuffer with
all of the color values of the current theme. Clients receive this with
the response to their initial WindowServer::Greet handshake.
When the theme is changed, WindowServer tells everyone by sending out
an UpdateSystemTheme message with a new SharedBuffer to use.
This does feel somewhat bloated somehow, but I'm sure we can iterate on
it over time and improve things.
To get one of the theme colors, use the Color(SystemColor) constructor:
painter.fill_rect(rect, SystemColor::HoverHighlight);
Some things don't work 100% right without a reboot. Specifically, when
constructing a GWidget, it will set its own background and foreground
colors based on the current SystemColor::Window and SystemColor::Text.
The widget is then stuck with these values, and they don't update on
system theme change, only on app restart.
All in all though, this is pretty cool. Merry Christmas! :^)
Allow everything to be built from the top level directory with just
'make', cleaned with 'make clean', and installed with 'make
install'. Also support these in any particular subdirectory.
Specifying 'make VERBOSE=1' will print each ld/g++/etc. command as
it runs.
Kernel and early host tools (IPCCompiler, etc.) are built as
object.host.o so that they don't conflict with other things built
with the cross-compiler.
4KB gets pretty mmap/munmap heavy when downloading larger files,
so bump this a bit to reduce time spent in memory allocation.
This can be improved in various ways, but I'm not sure what the
best way forward is at the moment.
This patch exposes some fields about purgeable memory regions.
We now also show total purgeable volatile and non-volatile memory in
the big process table.
Using int was a mistake. This patch changes String, StringImpl,
StringView and StringBuilder to use size_t instead of int for lengths.
Obviously a lot of code needs to change as a result of this.
This patch introduces code generation for the WindowServer IPC with
its clients. The client/server endpoints are defined by the two .ipc
files in Servers/WindowServer/: WindowServer.ipc and WindowClient.ipc
It now becomes significantly easier to add features and capabilities
to WindowServer since you don't have to know nearly as much about all
the intricate paths that IPC messages take between LibGUI and WSWindow.
The new system also uses significantly less IPC bandwidth since we're
now doing packed serialization instead of passing fixed-sized structs
of ~600 bytes for each message.
Some repaint coalescing optimizations are lost in this conversion and
we'll need to look at how to implement those in the new world.
The old CoreIPC::Client::Connection and CoreIPC::Server::Connection
classes are removed by this patch and replaced by use of ConnectionNG,
which will be renamed eventually.
Goodbye, old WindowServer IPC. You served us well :^)
This patch adds these I/O counters to each thread:
- (Inode) file read bytes
- (Inode) file write bytes
- Unix socket read bytes
- Unix socket write bytes
- IPv4 socket read bytes
- IPv4 socket write bytes
These are then exposed in /proc/all and seen in SystemMonitor.
Previously it was not possible to see what each thread in a process was
up to, or how much CPU it was consuming. This patch fixes that.
SystemMonitor and "top" now show threads instead of just processes.
"ps" is gonna need some more fixing, but it at least builds for now.
Fixes#66.
SystemServer can now create sockets on behalf of services before spawning any
of them, and pass the open socket fd as fd 3. CLocalServer gains a method to
complete the takeover and listen on the passed fd.
This is not used by any services at the moment.
Client-side connection objects must now provide both client and server
endpoint types. When a message is received from the server side, we try
to decode it using both endpoint types and then send it to the right
place for handling.
This now makes it possible for AudioServer to send unsolicited messages
to its clients. This opens up a ton of possibilities :^)
When adding a widget to a parent, you don't always want to append it to
the set of existing children, but instead insert it before one of them.
This patch makes that possible by adding CObject::insert_child_before()
which also produces a ChildAdded event with an additional before_child
pointer. This pointer is then used by GWidget to make sure that any
layout present maintains the correct order. (Without doing that, newly
added children would still be appended into the layout order, despite
having a different widget sibling order.)
This patch adds a limit of 200 unsent messages per client. If a client
does not handle its incoming messages and we manage to queue up 200
messages for it, we'll now disconnect that client. :^)
If an IPC client is giving us EAGAIN when trying to send him a message,
we now queue up the messages inside the CoreIPCServer::Connection and
will retry flushing them on next post/receive.
This prevents WindowServer from freezing up when one of its clients is
not taking care of its incoming messages.
Ports/.port_include.sh, Toolchain/BuildIt.sh, Toolchain/UseIt.sh
have been left largely untouched due to use of Bash-exclusive
functions and variables such as $BASH_SOURCE, pushd and popd.
This patch adds three separate per-process fault counters:
- Inode faults
An inode fault happens when we've memory-mapped a file from disk
and we end up having to load 1 page (4KB) of the file into memory.
- Zero faults
Memory returned by mmap() is lazily zeroed out. Every time we have
to zero out 1 page, we count a zero fault.
- CoW faults
VM objects can be shared by multiple mappings that make their own
unique copy iff they want to modify it. The typical reason here is
memory shared between a parent and child process.
This makes it so that "on_connected" always gets called first.
Since accepted sockets are connected before construction, they have
to manually set CSocket::m_connected.
GEventLoop was just a dummy subclass of CEventLoop anyway. The only
thing it actually did was make sure a GWindowServerConnectionw was
instantiated. We now take care of that in GApplication instead.
CEventLoop is now non-virtual and a little less confusing. :^)
Okay, I've spent a whole day on this now, and it finally kinda works!
With this patch, CObject and all of its derived classes are reference
counted instead of tree-owned.
The previous, Qt-like model was nice and familiar, but ultimately also
outdated and difficult to reason about.
CObject-derived types should now be stored in RefPtr/NonnullRefPtr and
each class can be constructed using the forwarding construct() helper:
auto widget = GWidget::construct(parent_widget);
Note that construct() simply forwards all arguments to an existing
constructor. It is inserted into each class by the C_OBJECT macro,
see CObject.h to understand how that works.
CObject::delete_later() disappears in this patch, as there is no longer
a single logical owner of a CObject.
We were only deleting the pointee when the ObjectPtr was destroyed.
If the ObjectPtr is cleared before that, we should also delete the
pointee. This is not the most important class to get right, since
it will go away as soon as we're able to switch to RefPtr.
It's pretty confusing when a CObject is owned both by its parent-child
relationship, but also by an ObjectPtr member in the parent.
In those cases, we have to make sure we both unparent the child *and*
reove it from the ObjectPtr.
This will become a bit less confusing when ObjectPtr becomes RefPtr,
although still not crystal clear. I'm not sure what the solution is.
Subclasses of CNetworkJob handle this by overriding shutdown().
This patch implements it for CHttpJob by simply tearing down the
underlying socket.
We also automatically call shutdown() after the job finishes,
regardless of success or failure. :^)
The C_OBJECT macro now also inserts a static construct(...) helper into
the class. Now we can make the constructor(s) private and instead call:
auto socket = CTCPSocket::construct(arguments);
construct() returns an ObjectPtr<T>, which we'll later switch to being
a NonnullRefPtr<T>, once everything else in in place for ref-counting.
With this patch, CEvents no longer stop at the target object, but will
bubble up the ancestor chain as long as CEvent::is_accepted() is false.
To the set accepted flag, call CEvent::accept().
To clear the accepted flag, call CEvent::ignore().
Events start out in the accepted state, so if you want them to bubble
up, you have to call ignore() on them.
Using this mechanism, we now ignore non-tabbing keydown events in
GWidget, causing them to bubble up through the widget's ancestors. :^)
Long-term we should use reference counting for the CObject hierarchy.
Since we've already accumulated a fair amount of code, this is quite a
large task, so I'm breaking it into some steps.
So, ObjectPtr is a "smart" pointer for CObject-derived types.
It becomes null when moved from, and will destroy unparented CObjects
in its destructor.
The idea here is to convert the codebase over to ObjectPtr piece by
piece, and then when everything is moved and CObject itself refactored
for ref-counting, we can just replace ObjectPtr with RefPtr everywhere.
RPC clients now send JSON-encoded requests to the RPC server.
The connection also stays alive instead of disconnecting automatically
after the initial CObject graph dump.
JSON payloads are preceded by a single host-order encoded 32-bit int
containing the length of the payload.
So far, we have three RPC commands:
- Identify
- GetAllObjects
- Disconnect
We'll be adding more of these as we go along. :^)
Both overloads should know how to set up a notifier callback in case
we get EINPROGRESS from connect().
It might be even better to merge the connect() overloads into a single
function..
We were returning a zero-length ByteBuffer in some cases. We should be
consistent about this and always return a null ByteBuffer if nothing
was read at all.
This was a workaround to be able to build on case-insensitive file
systems where it might get confused about <string.h> vs <String.h>.
Let's just not support building that way, so String.h can have an
objectively nicer name. :^)
The Inspector app quickly exposes crappy flat object hiearchies without
parent/child relationships. This is one of many commits that improves
the situation by making parent/child CObject relationships explicit.
All programs that have a CEventLoop now allow local socket connections
via /tmp/rpc.PID and will dump a serialized JSON array of all the live
CObjects in the program onto connecting sockets.
Also added a small /bin/rpcdump tool that connects to an RPC socket and
produces a raw dump of the JSON that comes out.
Now there's just CHttpRequest::set_url(URL), no need to specify the
host, port and path manually anymore.
Updated ChanViewer and Downloader for the API change.
This ensures the pipe fds don't leak into child processes.
This manifested as the Shell (and all processes started
from the shell) having two mysterious FIFOs open. This
was happening because of the Terminal, which the shell
was spawned form, leaking its CEventLoop wake pipe fds.
Fork the IPC Connection classes into Server:: and Client::ConnectionNG.
The new IPC messages are serialized very snugly instead of using the
same generic data structure for all messages.
Remove ASAPI.h since we now generate all of it from AudioServer.ipc :^)
This API was returning a "const char*" and it was unclear who took care
of the underlying memory. Returning a String makes that obvious.
Also make sure we close the /etc/passwd file when we're done with it.
There's some confusion between the write syscall and CIODevice::write()
here. The internal write() returns a boolean, and has already whined
in case the syscall failed, so we don't need to do that again.
The goal here is to generate most of this code from IPC protocol
descriptions, but for now I've spelled them all out to get started.
Each message gets a wrapper class in the ASAPI_Client or ASAPI_Server
namespace. They are convertible to and from the old message structs.
The real hotness happens when you want to make a synchronous request
to the other side:
auto response = send_sync<ASAPI_Client::GetMainMixVolume>();
Each request class knows his corresponding response class, so in the
above example, "response" will be an ASAPI_Server::DidGetMainMixVolume
object, and we can get the volume like so:
int volume = response.volume();
For posting messages that don't expect a response, you can still use
post_message() since the message classes are convertible:
post_message(ASAPI_Server::DidGetMainMixVolume(volume));
It's not perfect yet, but I already really like it. :^)
Processes can now have an icon assigned, which is essentially a 16x16 RGBA32
bitmap exposed as a shared buffer ID.
You set the icon ID by calling set_process_icon(int) and the icon ID will be
exposed through /proc/all.
To make this work, I added a mechanism for making shared buffers globally
accessible. For safety reasons, each app seals the icon buffer before making
it global.
Right now the first call to GWindow::set_icon() is what determines the
process icon. We'll probably change this in the future. :^)
We're going to be using dedicated server socket classes instead.
This was only implemented for CLocalSocket, and clients have been switched
over to using CLocalServer.
Use CLocalServer to listen for connections in WindowServer and AudioServer.
This allows us to accept incoming CLocalSocket objects from the CLocalServer
and construct client connections based on those.
Removed COpenedSocket since it's replaced by CLocalSocket.
Instead of trying to support both client and server in CLocalSocket, let's
have a specialized server class.
The basic usage is:
CLocalServer server;
server.listen("/tmp/name-of-portal");
server.on_ready_to_accept = [&] {
CLocalSocket* client = server.accept();
...
};
This will make things a lot simpler, since an accepting socket doesn't need
half of the stuff that a regular CIODevice provides. :^)
Since ChildAdded events originate from the CObject constructor, they are not
fully constructed when their parent learns that they were added.
Added a little comment about this to the child_event() declaration.
This macro goes at the top of every CObject-derived class like so:
class SomeClass : public CObject {
C_OBJECT(SomeClass)
public:
...
At the moment, all it does is create an override for the class_name() getter
but in the future this will be used to automatically insert member functions
into these classes.
If we had already processed a couple of queued events by the time we were
told to un-nest the event loop, we'd put the entire current batch at the
head of the outer queue. This meant that we might end up trying to process
the same events multiple times.
Let's not do that. :^)
We were installing libraries into /Libraries/Root, rather than in /Root.
This made the ports system behave rather unpredictable, since I had old
versions of things in /Root and new versions of things in /Libraries/Root.
Add a trivial CSafeSyscall template that calls a callback until it stops
returning EINTR, and use it everywhere we use select() now.
Thanks to Andreas for the suggestion of using a template parameter for
the syscall function to invoke.
Instead of LibGUI and WindowServer building their own copies of the drawing
and graphics code, let's it in a separate LibDraw library.
This avoids building the code twice, and will encourage better separation
of concerns. :^)
Sticking these in a namespace allows us to use a more generic
("Connection") term without clashing, which is way easier to understand
than to try to come up with unique names for both.
As a consequence, move to use an explicit handshake() method rather than
calling virtuals from the constructor. This seemed to not bother
AClientConnection, but LibGUI crashes (rightfully) because of it.
This way, CNotifier can mutate state to its little heart's content
without destroying the world when the global CNotifier hash changes
during delivery.
If custom I/O is being done outside CIODevice, we need a way to force blocking sometimes.
This also fixes the default of CLocalSocket to be non-blocking, the same
as CTCPSocket.
This patch generalizes the concept used in Piano to wake up the event loop
so it can react to something happening on a secondary thread.
Basically, there's a pipe who is always part of the file descriptor set we
pass to select(), and calling wake() simply writes a little to that pipe.
* Add a LibAudio, and move WAV file parsing there (via AWavFile and AWavLoader)
* Add CLocalSocket, and CSocket::connect() variant for local address types.
We make some small use of this in WindowServer (as that's where we
modelled it from), but don't get too invasive as this PR is already
quite large, and the WS I/O is a bit carefully done
* Add an AClientConnection which will eventually be used to talk to
AudioServer (and make use of it in Piano, though right now it really
doesn't do anything except connect, using our new CLocalSocket...)
Don't process any more events. We already prepend the remaining events in
this loop to the outer loop if needed.
If there were any more events queued after the exit request, the iteration
code would make an invalid access into 'queued_events'.
Fixes#300.
