Slimmer Async With Smol

While working on reworking Recordbox’s database and library parsing code to use diesel and parse files on a per-directory basis, respectively¹, I would eventually find myself inadvertently reworking how async code is run throughout the code², which resulted in an overall simplification of the execution of async tasks throughout Recordbox’s codebase. These improvements were born out two things. One was the spontaneous decision to switch the async runtime used by Recordbox from tokio to smol. The other was a discovery³ made while trying to debug issues with database lock contention, that led me to completely remove the async runtime entirely and rely almost entirely on the glib main loop as the single persistent async executor in the codebase. While any potential performance improvements may be subjective, the code itself is now undeniably simpler―with significantly less overhead when async is involved―and startup times, at least, seem to be slightly faster.

A Bit of Context #

Before jumping into all this, I’ll briefly cover some general context for asynchronous programming and how it pertains to Recordbox specifically.

The important context for async, or asynchronous programming, is that it is a concurrency model where tasks can pause without blocking the thread they are running on. One significant example of where this can be useful is for networking. An application and/or server, whether sending or receiving data on a network connection, can use asynchronous programming to wait on the results of a request, or wait for incoming requests, without having to spawn a thread for each connection or request. A more specific example is that, if your application makes an API request, it can wait for that request to complete while simultaneously doing other tasks without needing multiple threads. More generally, the async model can be applied to pretty much any kind of I/O, with some caveats depending on the type of I/O.

Async in Recordbox #

In the case of Recordbox, the majority of the async code is used to manage access to the database; parse music files in the background during a library sync; and load, scale, and cache thumbnails; all without blocking the UI. This is crucial. All GTK code in GTK applications run inside the MainLoop from glib, which runs single-threaded and all GTK code can only be called in the thread which is running this event loop. If you read from the database in the main loop thread without any sort of threading or async, the thread blocks until the read is complete, and the event loop cannot process any other events. While the thread is blocked, no user input can be processed, UI updates will not occur, and the application will not respond to external signals or events. If it blocks too long, the desktop will consider it unresponsive and ask the user if they want to terminate it.

With asynchronous programming, however, the main loop can process other events while an async task is running, and return to the task when action needs to be taken. In Rust and other languages, the points at which control can be released from the current task are denoted by the await keyword.

The await keyword can be used only inside functions or code blocks denoted with the async keyword. To generalize over the details, this changes the function into a Future. A Future is some task, sequence of code, etc, that completes “at some point”. Typically, a Future is inert until it is called with await, at which point it runs to completion. While it is running, however, the caller can do other things. Notably, it can run other asynchronous tasks.

In the previous example, where I mentioned that accessing the database will block the thread until the query is complete, there is no asynchronous programming in use. To make it async, a common method—and the method used in Recordbox—is to use a connection pool, which allows for some number of connections to the database, and creates a separate thread for each connection. When the main loop in the application needs to access the database, it retrieves one of these connections asynchronously, and then provides a closure with the code to retrieve the desired data from the connection. These tasks are Futures, so these methods are called with await, whereupon they will begin to execute their code. At this point, where a future has been awaited but it hasn’t finished yet, the main loop can switch to normal event processing. It will poll any running futures to check if they are complete, or have a mechanism for futures to signal when they have finished, and once a running future completes, the control flow of the application returns to whatever is next after the await statement.

For a quick shorthand, an await can be thought of to mean “run this code, which will be waiting until some external task completes and not otherwise doing anything, so other things can be run in the meantime until it completes.” This abstracts over quite a bit of specifics about how async code is run which is not particularity relevant in this post.

Executors and Runtimes #

Async code cannot run without something to, well—run it. Typically, this is done by an Executor.

Swapping Out Tokio #

While reworking how directories are are scanned and music is parsed in Recordbox I was making significant structural changes to async code in particular. This brought the asynchronous parts of Recordbox back into focus and I started thinking about improvements I could make beyond the changes I was already doing. For a while I was somewhat bothered by the fact that tokio’s async runtime spawns a thread per core to drive async tasks, but I couldn’t necessarily switch to a LocalRuntime because the point of the async runtime was to offload costly tasks out of the application thread. It’s been a while since I was messing around with tokio, but I remember something along the lines of not being able to send/receive tasks to a LocalRuntime which was started on another thread, and starting another async runtime on the application thread was pointless when the glib main loop could be used instead. I could configure the runtime to use fewer threads, but overall I had this feeling that using tokio at all was somewhat overkill for my purposes.

At some point, I ended up reading about smol, which is a modular collection of various async crates to provide an alternative to tokio, and a “build your own executor” approach. So I decided to switch to it. This was actually quite straightforward; just swapping out tokio imports for smol imports and renaming some tokio functions like spawn_blocking to the smol equivalent. Things like RwLocks and Mutexes turned out to have the same interface between the two, so just swapping out use tokio::sync::Mutex for use smol:🔒:Mutex was enough.

The result was a much smaller runtime: an async Executor running on a single thread, separate from the main application loop, that async tasks could be spawned on to offload work from the application thread.

Really though, this didn’t change much other than to reduce the number of threads being used and shuffle some of the project’s dependencies. It was nearly an entirely lateral switch for minor, if any, advantages.

But then, while desperately trying to debug lock contention on the database and making changes to the code to avoid it, I stumbled into something that led to me throwing out the entire Executor. And that something was the ability use block_on in the the database connection threads.

Removing The Executor #

In retrospect, the ability to use block_on in connection threads is quite obvious, but in the past when trying to figure out why the MprisServer would sometimes fail to start correctly I ended up running into the problem that I couldn’t use block_on in the context of another async executor. Since everything was running inside the context of tokio, it would panic and crash the application. Because of the amount of frustration that this had caused at the time, I just ended up not using block_on ever.

Two things were different now, though. One: the block_on from smol is not the same one as the one from the futures crate, which I was using before. The one from smol can be nested in as many async contexts as you want. Two: the entire application is no longer running inside a tokio runtime.

Ironically, I don’t think either of these things were actually relevant, because the threads which access the database in the connection pool are spawned separately from the tokio runtime, so even the old block_on would work even if I hadn’t ditched the tokio runtime. Also, I mostly discovered this as an attempt to fix the lock contention problem with the database, which it didn’t fix, and in fact the problem was fixed entirely independently of any of this. Also ironically, the original block_on statement which I added that resulted in the discovery isn’t even there anymore! But regardless, this sequence of events led me to discover this, so I guess I’ll take it.

Why It Was Possible #

Originally, the LibraryManager, which scans the music directory to check for changes and parse new or modified files before saving them in the database, would have access to the entire MusicDB struct managing the database connection pool, and retrieve a connection whenever it needed to read or write. In an attempt to solve the issue I was facing, where connections would fail because the database was locked, I changed it so the entire LibraryManager had access to only one database connection. This meant that the sync performed by said library manager—which was async—had to be run by the thread with the connection, which wasn’t. I couldn’t call spawn, to run it on the executor either, because to get the return value from it, it would need to be awaited. This was, again, async.

Since I was so frustrated with the troubleshooting, I decided to just throw the async task in a block_on and hope it worked. I didn’t have high hopes, because the previously mentioned bad experiences with it had biased me against it.

But it worked. The result was that all of the async code which was run during a sync was managed by the block_on executor in the thread which owned the database connection, instead of the global executor I was using before.