வரிசை · queue, row, order

Per-key message queues — ordering from the key, not a partition count

Varisai is a durable message log with per-message delivery tracking. Messages with the same key arrive strictly in order; different keys never wait for each other. Set key = tenant_id and a noisy neighbour becomes structurally impossible for you.

Open the dashboard Read the API docs

By Higglers Lab, the team that builds Varisai · Last updated 2026-10-08

# publish — the key is the whole model
curl -X POST https://varisai.dev/v1/streams/orders/messages \
  -H "Authorization: Bearer vk_yourkey:vs_yoursecret" \
  -d '{"key":"tenant_42", "payload":{"order_id":8891}, "deliver_at":"2026-09-01T10:00:00Z"}'

# consume — retries, backoff, and DLQ are declared, not coded
const stop = varisai.consume("billing", async (message) => {
  await chargeCustomer(message.payload); // throw = retry, resolve = ack
});

What is Varisai?

Varisai is a self-hosted message broker: a durable message log with per-message delivery tracking, where ordering and parallelism are governed by the message key rather than by a partition count chosen in advance. Messages with the same key are delivered strictly in order, one at a time per subscription; messages with different keys never wait for each other.

We built it because every team we worked on rebuilt the same glue around Kafka or RabbitMQ: a retry loop, a dead-letter table, a cron job for delayed work, and a trace store for "why wasn't this processed?". In Varisai those are broker features you declare once on a subscription. It runs on Node.js and MongoDB, and the source is public under the Elastic License 2.0.

Type: Durable message log with per-key delivery lanes
Ordering unit: The message key: strict FIFO per (stream, key, subscription)
Delivery: At-least-once, leased per attempt (30 s lease by default)
Retries: 5 attempts by default, exponential backoff with jitter from 10 s up to 1 h
Retention: 7 days per stream by default; 30 days for dead-lettered messages
Limits: 1 MB per message and 500 in-flight deliveries per subscription by default; all configurable
Consume API: HTTP long-poll, up to 25 s per receive, up to 100 messages per call
Storage: MongoDB replica set, publishes acknowledged with majority write concern
License: Elastic License 2.0 — free to use, self-host, and modify

What do you get out of the box?

Kafka is a log without per-message delivery. RabbitMQ delivers but forgets. Teams bolt Postgres and Redis alongside either to fill the gap — Varisai is that gap, as one system.

Per-key lanes

Strict FIFO per key with unbounded parallelism across keys. A stuck key parks its own lane and blocks nothing else — head-of-line blocking is gone by construction.

Declared retries & DLQ

Max attempts, exponential backoff with jitter, per-attempt leases, and a browsable dead-letter queue with one-click redrive — configured on the subscription, not coded in every consumer.

Scheduled delivery

Publish with deliver_at and the message stays invisible until that instant — durable, cancellable until it fires, and visible on its own dashboard screen.

Trace any message

"Why hasn't this been processed?" is one paste away: every attempt, consumer, duration, and error — and if it's blocked, the exact message blocking it, by ID.

Replay one tenant

Acking never deletes. Seek any subscription to an offset or timestamp — optionally for a single key — and replay one tenant's history without touching anyone else's.

An operational UI

Not a stats dashboard — a tool. Diagnose a stuck consumer, skip a poison message, redrive a failure group, and pause a lane without leaving the browser.

How does the key model work?

Every message carries a key. That single field replaces the partitions, queues, and sharding decisions you would otherwise make up front.

Same key

Delivered strictly in publish order, one at a time per subscription.
A failing message parks its lane; everything behind it waits — as it should.
Effective serial pipeline when you want one: use one key.

Different keys

Fully independent: no shared cursor, no shared failure.
Parallelism = number of active keys, not a config value.
A million keys and a thousand consumers is the same object as one key.

Two scenarios, one primitive

Everything below is a single feature — the message key decides what waits and what runs in parallel. Both animations loop through a real failure.

Ticket booking — first come, first served

Five buyers, four tickets, key = event_id: one lane, one buyer processed at a time — overselling is impossible by construction. When P1's reservation keeps failing, the retry policy backs off while P2–P5 hold their exact places; after max attempts P1 dead-letters, the lane resumes, and the remaining four tickets sell in arrival order.

Figure 1: one event, one lane — buyers are served in arrival order and a failing buyer cannot block the sale.

key = event_idretry: exponential, then DLQdeliver_at = hold expiry

Ride matching — dispatch without double-booking

Ten riders, five drivers, two keys. key = zone queues riders first-come-first-served; key = driver_id means a driver can never hold two offers at once. When every driver is busy, C6 waits at the head of the lane — retrying, never losing their place — and takes the first driver who frees up, before C7. Runnable in examples/ride-matching.

Figure 2: two keys — riders queue per zone, drivers hold one offer at a time, and a waiting rider keeps their place.

key = zone → rider FCFSkey = driver_id → one offer in flightnack + retry = waiting for a cab

How does Varisai compare to Kafka, SQS FIFO, and RabbitMQ?

All four are good systems that optimise for different things. The question that separates them: where does ordering come from, and what happens to a message after it fails?

Sources: the Kafka documentation on partition ordering, the AWS guides on SQS delay queues and SQS FIFO quotas, and the RabbitMQ queue documentation.
	Varisai	Apache Kafka	Amazon SQS FIFO	RabbitMQ
Ordering unit	The message key, declared per message	The partition, sized up front	Message group ID	The queue (one consumer for strict order)
Failing message blocks…	Only its own key's lane	Its whole partition	Its whole message group	The queue, or goes to the back
Retries and DLQ	Declared per subscription, browsable, one-call redrive	Build it (retry topics)	Redrive policy, no backoff shaping	TTL + dead-letter exchange plumbing
Scheduled delivery	Any future time; cancellable	Not built in	Delay up to 15 min	Plugin or TTL tricks
Replay after ack	By offset or time, even for one key	By offset, whole partition	Impossible	Impossible

On the other hand, if you move hundreds of megabytes per second of telemetry to many independent readers, Kafka is the right tool and Varisai is not. Varisai tracks delivery state per message; that bookkeeping is the product, and it has a cost. The full comparison lists when you should not pick it.

Which guarantees are enforced, and which are not?

What is promised is enforced. What is not promised is written down.

Guaranteed

Durability: publishes acknowledged with majority writes.
Strict FIFO per (stream, key, subscription).
At-least-once delivery until acked, dead-lettered, or expired.
Retention independent of acking; replay by offset, time, or key.
Scheduled delivery that survives restarts.

Not guaranteed

No exactly-once — consumers must tolerate a repeat. Dedup narrows the window; it cannot close it.
No ordering across different keys.
No cross-stream transactions.
Scheduled delivery is "at or after", not "exactly at".

"Nobody honestly has exactly-once side effects or cross-stream transactions. Varisai does not pretend to."— Higglers Lab, from the Varisai comparison page

Frequently asked questions

The short answers. The full FAQ covers crashes, leases, dedup, and production setup.

Why not just use Kafka with a retry topic?

You can, and many teams do. On the other hand, the retry topic, the dead-letter topic, the delayed-redelivery service, and the trace store are four more things you own, and one bad message can still stall its whole partition.

Varisai makes those broker features that you declare on a subscription. If you already run Kafka happily at firehose throughput, keep it; Varisai is for the per-entity work queues beside it. See the full comparison.

What happens when a message keeps failing?

Each failed attempt is retried with the subscription's backoff (exponential with jitter by default, from 10s up to 1h) until max attempts (5 by default). Then the message is dead-lettered: its lane resumes, and you can browse the failure, group it by error, and redrive it in one call.

How do you replay one tenant without touching the others?

Acking never deletes. Seek the subscription to an offset or a timestamp and pass the key you care about; only that key's lane rewinds, and every other key keeps flowing from where it was.

Is delivery exactly-once?

No, and we say so up front. Delivery is at-least-once until a message is acked, dead-lettered, or expired. Publish deduplication with an idempotency_key narrows the window; your consumer still has to tolerate a repeat.

When should you not use Varisai?

When you need raw firehose throughput to many independent readers (Kafka's home game), when you are all-in on AWS and SQS FIFO's 15-minute delay cap never hurts, or when you need complex routing topologies (RabbitMQ's exchanges and bindings). Varisai tracks delivery state per message; that bookkeeping is the product, and it has a cost.

Key takeaways

The key is the whole model. Same key = strict order; different keys = full parallelism. Nothing to size or rebalance.
Failures stay local. A failing message parks only its own lane, retries with backoff, then dead-letters.
Retries, DLQ, scheduling, and tracing are declared on the subscription, not coded in every consumer.
Acking never deletes. Replay any subscription by offset, time, or a single key.
Honest limits. At-least-once delivery, no cross-key ordering, no cross-stream transactions.
Self-hosted and free under the Elastic License 2.0, on Node.js and MongoDB.

How do you get started?

Web tier, worker, and MongoDB replica set — one docker compose up. Cognito sign-in, scoped API tokens, Prometheus metrics, and TLS overlay included.

Clone the repository and run docker compose up (Docker Compose starts the web tier, the worker, and a single-node MongoDB replica set).
For this local installation, open http://localhost:7700, create a project, and copy its API key.
Publish a message with a key, create a subscription with your retry policy, and consume — the 5-minute guide walks through each call.

Create an account Read the 5-minute guide

Not ready yet? Get an email when new releases land.