← notes
$ cat notes/darkslide-devlog-why-i-put-sqlite.md

Darkslide devlog: why I put SQLite in a cage.

/ 3 min read /
#darkslide#devlog#rust

What I had

The photo editor I build stores the data like LUTs or image adjustments in SQLite. The way database was implemented is simple — a Mutex wrapping a rusqlite::Connection, inside an Arc in AppState. So I can easily share the DB connection between multiple modules.

pub db_instance: Arc<Mutex<DB>>

Every time in order to use the connection it had to do this: lock().unwrap(), run the query, return the result. It worked fine until the DB had to be used in a concurrent part. Soon I will implement loading of adjustments from DB, but the opening process of images runs in parallel and it will need some access to DB. Also, usually working with SQLite should be done from the same thread, it does not handle parallel access out of the box.

So, I needed the DB to work like an operation queue — one dedicated thread owning the connection, everything else submitting jobs and getting callbacks when done. In the Apple world I’d use NSOperationQueue with maxConcurrentOperationCount = 1. In Rust, I wanted the same pattern but with async/await on top.

The requirements were:

  • One thread owns the SQLite connection

  • Callers send “requests” to that thread

  • Results come back via some kind of callback

  • Public API is async fn so commands can .await

After some research I had basically two options.

Option 1: Use an existing Crate

tokio-rusqlite does exactly this. I checked it first.

But the problem: it is kinda outdated. I would have to downgrade rustqlite and rustlite-migration since tokio-rustqlite was not updates for 8 months. Does not sound like a good choice. As the code of this crate is not actually that big.

Option 2: Build It Myself

Based on what I need and the code I read from tokio-rustqlite I went with this:

type Job = Box<dyn FnOnce(&mut Connection) + Send + ‘static>;
pub struct Db {
    tx: std::sync::mpsc::Sender<Job>,
}

One std::thread owns the Connection. It runs:

for job in receiver {
    job(&mut conn);
}

Basically it’s like Event loop or Actor loop, whatever you call it. Blocks the thread until next items comes from receiver.

The API is async fn. Each method sends a closure to the worker and .await`s a tokio::sync::oneshot reply:

async fn call<T, F>(&self, f: F) -> Result<T, String>
where
    F: FnOnce(&mut Connection) -> Result<T, String> + Send + ‘static,
    T: Send + ‘static,
{
    let (reply_tx, reply_rx) = tokio::sync::oneshot::channel();
    self.tx.send(Box::new(move |conn| {
        let _ = reply_tx.send(f(conn));
    }))?;
    reply_rx.await?
}

One thing: .await suspends the task, not the thread. The tokio worker is freed to run other commands. When the DB thread finishes and fires the oneshot, tokio runtime wakes the task on any available worker.

What I Learned

The biggest surprise was: holding a MutexGuard across .await. If you do something like let db = state.db_instance.lock().unwrap() followed by db.load_luts().await?, Rust gives you a clear compile error: future is not `Send`.

The fix isn’t a workaround — it’s the architecture: `Db` is no longer inside a `Mutex`, so there’s no guard to hold.

That is, actually, a very good thing. Compiler prevents me from a stupid mistake that could lead to hours of debugging.

Darkslide is keyboard-driven photo editor.

$ echo "test"