Tinkering with Event Sourcing and CQRS with Elixir

Yet Another Hobby Project

Last month I’ve read the awesome book “Real-World Event Sourcing” by Kevin Hoffman. Needless to say, I was hyped to start coding something like that with Elixir, and what luck–The book uses Elixir as well!

Before you point out that this kind of architecture isn’t always necessary, remember this is just a fun project to try out something new, so no hard feelings.

I’ve been wanting to create something related to backends in games for a while. The idea was to develop a full backend for a gacha game — managing users, the characters they own, allowing users to form teams of four characters from their collection, and, of course, the core gacha mechanic. Players would open packs of characters, and the system would handle the random draws behind the scenes.

The Initial Architecture

Initially, I was working on a very basic CRUD setup, using Phoenix generators to create entities left and right. Users, characters, items, packs, relationships between users and characters, ownership of packs, etc., were all just normal database entities. It became a bit tedious, so I decided to explore a different architecture.

I was already familiar with Event Sourcing and CQRS, but I’d never implemented them before. So, being the academic I am, I firsth read that book, did some research, and then jumped into the practical side.

Commanded to the Rescue

After reading the book, I finally had a clear understanding of how Event Sourcing and CQRS work, along with the core concepts. For Elixir, there’s a fantastic library called Commanded, which provides everything you need out of the box (Commands, Aggregates, Events, Projections, etc.).

After a bit of analysis paralysis over which database to use for the event store and projections, I settled on PostgreSQL for both. Commanded offers out-of-the-box support for both PostgreSQL and EventStore DB, but I chose PostgreSQL because I wanted more hands-on practice with it.

In fact, I’m using two separate PostgreSQL databases — one for the event store and another for projections. This separation keeps the Command (write) side and the Query (read) side distinct. This way I can actually implement CQRS with Event Sourcing.

The First Event Flow

For my first test, I set up a flow for user registration. The idea is that the user fills out a registration form on the frontend, submits it, and clicks “Create Account.” This calls a REST API built with Phoenix for user registration. The controller receiving the request validates the data (using Ecto changesets), and then dispatches the command to the User aggregate.

The aggregate emits a UserRegistered event, which is stored in the event store. This triggers a projection, which saves the user’s data to the query-side database. The projection writes the data to a users table, where we store things like the user ID, email, password hash, etc.

A Sneak Peek

I’ve built the registration form in Unity, to mimic a game client that could as well be built as a mobile game for smartphones.

(Ignore the ugly UI, I just used the default Unity UI elements withouth any styling)

Once the “Create Account” button is clicked or tapped, the client makes a REST API call to start the flow.

A Note on the Unity Client

The way the Unity client can call my API is actually via an SDK generated from the OpenAPI spec. I’m also maintaining an OpenAPI spec and using a generator to create the SDK in C# which I then install as a library in the Unity game. It’s pretty handy and makes the client-server communication a breeze.

The Backend Side

When the client sends the request, the flow kicks off and the command is dispatched. After successful validation, the UserRegistered event is emitted and stored in the event store (the PostgreSQL DB).

This, in turn, triggers the projection, saving the user’s data into the query database (the other PostgreSQL DB). Commanded makes it easy to create projections with its companion library commanded-ecto-projections.

At this point the user is registered and the API responds with an authentication token that the client can use to authenticate in the future. The flow is set with the “strong consistency” guarantee, so the client can be sure the registration was completed, and the system can issue the auth token before responding.

Next Experiments

Creating this flow was a lot of fun, and it helped me apply the principles of Event Sourcing and CQRS in a real project. I’ve got some more ideas I want to experiment with:

• Model the character pack opening process: I’ve already implemented it as a simple CRUD operation, so the logic is there. Moving it to this architecture will be more complex since it involves several moving parts.

• Integrate KeyCloak for user management: It’s been a valuable learning experience to implement user registration and authentication myself, but I want a more robust solution. Integrating KeyCloak will allow me to focus more on the game backend itself.

• Experiment with Apisix as the API Gateway: I’ve been reading about Apisix, and it seems to have great integration with KeyCloak. I’m considering placing it in front of my Phoenix API to handle authentication and improve security.

Stay tuned! When I make more progress, I’ll be sure to write another post.