This blog post is about the new design of my wayland libs. It's not a tutorial, but rather about the design choices I made, and why I made them.
I'll start with a quick recap of what wayland is, from an API & protocol point of view. If you already know that, you can directly jump to second section.
Then I'll list the constraints I have to work with regarding my API design. After that, I'll do a quick recap of my two previous design attempts, and then dig into my new design.
It'll be a long article, but I'll try to keep it well structured, so feel free to fast-forward to the parts that interest you the most, if you feel like it's TL;DR. 😉
What's Wayland (the protocol)?
I won't go into details about what Wayland is for, wikipedia does a good job already and this is not the focus of this post. What interests me here is how it works.
Wayland is a message-based and object-oriented protocol: the clients and the compositor (server) exchange messages on an unix socket, these messages being structured on instances of interfaces. Both ends can create objects in a shared namespace, an each message is associated with an object.
Each interface (class) defines a number of requests (messages from the client to the server) and
events (messages from the server to the client). Messages can create objects, and they are actually
the only way to create object. For example, an instance of
wl_compositor will allow the client to
create as many
wl_surfaces as they want, with the
There are special kind of objects called "global objects". The client can instanciate them at will
from an even-more-special object: the
wl_registry, which is the entry point of the protocols. The
registry will list to the client the global objects that the server have defined, and the client is
free to instanciate them as many time as they want and send requests (and receive events).
Note I've used a plural: protocols. While the core wayland protocol defines a number of globals, it is possible to anybody to use a protocol extension, by writing an XML file specifying it. As long as the client and server have both been built using this XML file, they can use the extension. This is determined in the initialisation, as the entry point of this protocol extension will be one more globals. The registry doesn't have these globals? That means the server does not support this extension. Simple.
The design constraints
Designing the API for Wayland on Rust was not an easy task, for several reasons:
Need to use the C libs
The socket protocol being specified, my first idea was to go full-rust, reimplement the whole protocol, and be as free as I want. But...
Mesa is built around the official C libraries. That means that if I want OpenGL and Vulkan to be usable, I must provide pointers that can be passed to mesa, that it'll use with the C library. So I must use the C library (or replicate exactly it's internal structure, but that'd be silly).
So I have to build around the C libraries, and the design choice they made. As unrusty as they can be:
- pointers everywhere, meaning I'll need to build a safe layer around this
- callback-based, meaning I risk lifetime nightmare if I'm not carefull enough (and I experienced it with my first design attempt)
- thread-safe, it's a good thing, but will raise a lot of conccurency questions
Need for code-generation
The protocols are defined in XML files. The C libs come with a tool called
uses these files to generate code. The actual ABI of the libs is very small, and only contains
message serialization primitives for the socket. All the rest is built by the protocol XML specification.
I'll obviously need to do the same, so I can easily handle upgrades of the xml spec, as well as protocol extensions.
I had this constraint:
- Allow the user to use protocol extensions without needing to patch my lib (my second attempt didn't allow that)
Exploit client-server symmetry
This is not a hard requirement a priori, but more a personal goal. There is a lot of symmetry in the protocol regarding client and server side. I can surely make good use of that symmetry in the code generation, as long as I find a design that can handle the few asymmetries:
- Most objects are created and destroyed by the client, meaning the server will most of the time have to handle objects without controlling their lifespan. This is somehow the absolute opposite of Rust's ownership story.
1st try: closures for callbacks
My first try (versions 0.1 and 0.2 of the libs) was very naive: hand-write the whole protocol, with a method for each event where the user provides a closure as a callback.
I quickly understood this could not work, because far too painfull to use: to meet the thread-safety
rules, I quickly had to put
'static bounds in a lot of places, making the whole
thing impratical as long as the callbacks needed to interact with a shared state, which is quite
common in this context. Had to put
Arc<Mutex<_>> everywhere, even in single-threaded applications.
Not a good design at all.
2nd try: event iterators
For a second try (versions 0.3 to 0.6) I chose to use a nice corner of the wayland API: there is a mechanism allowing to plug callbacks at a deeper level of the message dispaching, and bypass the whole callback hell by instead providing a single callback to get the raw messages and dispatch them manually.
I decided to use this mechanism to replace the callbacks by an iterator of events, which is much more Rust-friendly, but had a few drawbacks: it's no longer possible to provide references to the objects directly as args of the callback, so I had to use opaque IDs. Once receiving an event, the user had to check its id an compare it to their object store to figure out which object is concerned by the event. It was not very good, but it worked.
This approach was absolutely not possible to use for server-side, because of the lifetime issues. Given most objects are created and destroyed by the client, the server must follow things precisely. Especially, when the client destroys an object in a request, the server must destroy its handle of this object. Failure to do so will completely mess up the internal state of the C libraries and cause the rest of the message exchange to be completely corrupted.
Other issue, the event iterator return type is a large enum, that must be aware of all possible events. So it must be aware of all existing protcols. So using a protocol extension requires to patch the lib.
There is still room for improvement.
3rd try: Handler traits
That's what led me to the current new design (version 0.7):
- each interface defines a
Handlertrait, with a method for each event/request (depending on the side)
- the user create handler structs, that each implement one or more of these traits
- these handler structs are given to the event_queue / event_loop (name depending on the side)
- each wayland object is registered to a handler, whith appropriate type-checking to ensure the appropriate handler trait is implemented
The idea here is that the handler struct is the shared state, each method of a
Handler trait takes
&mut self argument, allowing to access a mutable state without the need of synchronization
primitives. The C libs allow me to do that, because I have the guarantee that the events are processed
sequentially: there will never be two events/requests dispatched concurrently.
Also, using the appropriate trait machinery (which I won't detail here), I got a very generic
setup, allowing for new protocol extensions to easily be used by the user, without needing to patch
the lib. They just need to setup a build script using the
wayland-scanner crate, and it'll generate
a module for the protocol that can directly integrate in the existing handling setup.
This is quite symmetric, and there are very few special cases in
wayland-scanner code depending
on the side (client/server) the code is generated for. More code reuse, check.
The destructor safety didn't totally go away, and I had to integrate a mechanism so that all handles
to a same wayland object (as I create them out of thin air for handler methods, there can be more
than one) share an
AtomicBool registering if the object was destroyed or not, and ignoring any
attempt to send a message with a destroyed object (and returning an error signifying the destroyed state).
What could be better
With specialization, I could have made the API much more elegant. For now, there are some macros that the user need to use to make handler work, and that are there only to write impls of traits that could have been achieved via specialization, I think.
There are still some things that I see as not elegant I will try to change as soon as I find a satisfying design. First of them, the fact that to handle remote-destruction of objects, all of their methods return a result-like enum, even objects that cannot be destroyed!
What will be blocking
I know this looks silly, but I need this feature, and I cannot do without it in the long term.
The core protocol is okay for now, but the is a protocol extension that needs to be supported, it's
XDG-shell and will be in near future (or is already actually) the standard for managing the
shell interaction of the client (create windows, move them, get them fullscreen, resize them, etc...).
The code generation forces me to generate a large graph of statics with pointers to each other. It's a binary representation of the interfaces and their interactions, that I must provide to the C libs to use the protocols extensions, and I can't change the format.
And the graph for
XDG-shell contains a cycle.
For now, my short term goals are:
wayland-kbdto the new design
- on top of them, update
winitto the new design
- then, start digging into servo on wayland, for fun, awesomeness, and intellectual interest
Also an interesting project would be to build a
wlc-like crate on
wayland-server, but this is not my priority.