Slot reference
A slot is defined as slot_NNN with a kind and that kind's options. This page describes what reads and writes do for each kind.
slot_002: # slot number 002
kind: token_bucket # one of the kinds below
bucket_size: 100 # options of that kind
period: second
| Kind | Read | Write | s / d |
|---|---|---|---|
simple_memory | Value | Stores value | 010 |
timeout_memory | Value | Stores value if owner or no owner, else 007 | 010 |
token_bucket | Takes tokens, returns how many | 006 | 010 |
leaky_bucket | 1 accepted / 0 rejected | 006 | 010 |
ticker | Current countdown | Restarts the countdown | 010 |
atomic | Increments, returns new value | Sets the counter | 010 |
broadcast | Last value written | Sends event to every client | 010 |
multicast | Last value written | Sends event to subscribers | Join / leave |
Every kind also accepts a users map with per-user permissions. See Authentication.
An unknown kind, a missing required option or an out-of-range value makes the server skip that slot without any warning. The slot then answers e<slot>005 (MISSING_SLOT). Read each slot once after a configuration change to make sure it exists.
Simple memory#
kind: simple_memory. The most basic slot: a value of up to 36 bytes that anyone with permission can read and write. It has no options.
slot_000:
kind: simple_memory
- A read returns the current value. A slot that was never written returns an empty value (
v000). - A write replaces the value and echoes it back.
w000with no value empties the slot.
Timeout memory#
kind: timeout_memory. A memory slot with an owner. Only the connection that last wrote the slot can write it again, until it goes timeout seconds without writing. Use it as a lease to pick a single active worker or leader.
| Option | Required | Description |
|---|---|---|
timeout | Yes | Ownership timeout in seconds. Must be at least 1. |
slot_001:
kind: timeout_memory
timeout: 10
- If the slot has no owner, or the owner's timeout has expired, a write succeeds and the writer becomes the owner.
- A write by the owner succeeds and restarts its timeout.
- A write by any other connection fails with
007(WRITE_FAILED). - Anyone with read permission can read the value. Reads don't extend ownership. The value stays readable after ownership expires.
Captured with two nc sessions, A and B, against timeout: 10:
# A takes the lease
> w001node-a
< v001node-a
# B tries and is refused, but can read the owner
> w001node-b
< e001007
> r001
< v001node-a
# A closes its connection; B is still refused
> w001node-b
< e001007
# 10 seconds later
> w001node-b
< v001node-b
Ownership caveats#
- The owner is the TCP connection, not the user or the host. Two connections from the same process or the same user are two different owners.
- A lost connection doesn't release the slot. Only the timeout does. A client that reconnects right after a drop is a new owner, and its writes fail with
007for the rest of the timeout. The timeout is effectively the recovery time after a crash, so pick it with that in mind. - There is no release command. An owner that wants to hand over can only stop writing and let the timeout expire.
- A Ghoti restart forgets the owner. State is in memory only, so after a restart the first connection to write becomes the owner, even if another client still believes it holds the lease.
- There is no fencing token. An old owner that was paused (for example by a long GC pause) and has not noticed that its lease expired can still act on its old belief.
- Over HTTP every request is its own connection, so no HTTP client can own the slot across two writes. Use the
standardortelnettransport.
Token bucket#
kind: token_bucket. A classic token bucket for rate limiting. A fleet of API servers can share one bucket to enforce a single limit on calls to an upstream.
| Option | Required | Description |
|---|---|---|
bucket_size | Yes | Maximum number of tokens the bucket holds. At least 1. |
period | Yes | Refill period: second, minute or hour. |
refresh_rate | No, default 1 | Tokens added every period. Between 1 and bucket_size. |
tokens_per_req | No, default 1 | Most tokens handed out by one read. Between 1 and bucket_size. A client can take several tokens per call and use them locally, which cuts the number of requests. |
slot_002:
kind: token_bucket
bucket_size: 200 # allow bursts of up to 2x
period: second
refresh_rate: 100 # 100 tokens per second
- A read takes
tokens_per_reqtokens, or as many as are left if fewer, and returns how many it took.0means the bucket is empty. - The bucket starts with
refresh_ratetokens (one period's worth), not full. refresh_ratetokens are credited for every period that has passed, up tobucket_size, so tokens accumulate while the bucket is idle. Periods are aligned to the clock: they start at the beginning of each second, minute or hour, not at the time of the first read.- Writes are not allowed. Every write is rejected with
006(WRITE_PERMISSION), or403over HTTP. - Every read consumes tokens, so a read is not safe to retry.
Leaky bucket#
kind: leaky_bucket. A bucket that leaks one token every refresh_rate milliseconds. Each read tries to add a token: if there is room, the request is accepted. Once the bucket is full, requests are rejected until a token leaks out. The bucket size allows bursts, after which requests are accepted at the constant leak rate. Set bucket_size: 1 to allow no burst.
| Option | Required | Description |
|---|---|---|
bucket_size | Yes | Capacity of the bucket. At least 1. |
refresh_rate | No, default 1000 | Milliseconds between leaked tokens. At least 1. |
slot_003:
kind: leaky_bucket
bucket_size: 100
refresh_rate: 1000
- A read returns
1if the request was accepted and0if it was rejected. - Writes are rejected with
006(WRITE_PERMISSION). - Every accepted read fills the bucket, so a read is not safe to retry.
Ticker#
kind: ticker. A watchdog countdown, as in embedded circuits and microcontrollers. The slot holds a non-negative integer that goes down by one every refresh_rate milliseconds until it reaches zero. Writing a value restarts the countdown from that value.
| Option | Required | Description |
|---|---|---|
initial_value | Yes | Starting value. Must not be negative. |
refresh_rate | Yes | Milliseconds per tick. At least 1. |
Both options are required. A ticker without either one is ignored and answers 005, like any invalid slot.
slot_004:
kind: ticker
initial_value: 600
refresh_rate: 1000
- With
refresh_rate: 1000, writing600and reading 9 minutes later returns about60. One minute after that it returns0. - The countdown is evaluated lazily: a read applies all the ticks that elapsed since the previous read. Reading more often doesn't make it go faster.
- Ticks are aligned to multiples of
refresh_rateon the clock, so the first tick after a write can come sooner thanrefresh_rate. - A write must be a non-negative decimal integer. Anything else is rejected with
007.
Atomic counter#
kind: atomic. An integer counter that only goes up. Use it to hand out distinct numbers to concurrent workers. It has no options.
slot_005:
kind: atomic
- A read increments the counter and returns the new value. A new slot returns
1on the first read,2on the second, and so on. The counter never decrements. - A write sets the counter to a non-negative decimal integer, so after a write of
5the next read returns6. A negative value or anything that isn't an integer is rejected with007. - When the counter reaches the largest signed 64-bit integer, the next read wraps it back to
0. - Every read has a side effect, so a read must never be retried blindly.
- The counter is in memory: a restart resets it to 0. Don't use it where numbers must stay unique across restarts.
> r005
< v0051
> r005
< v0052
> w00541
< v00541
> r005
< v00542
> w005-1
< e005007
Broadcast#
kind: broadcast. A write is sent as an async event to every connected client, including the one that wrote it. No subscription is needed. Use it to push a signal, such as an ID or a "reload configuration" notice, to every instance. It has no options apart from users.
slot_006:
kind: broadcast
The event is a, the slot number and the value. The writer gets its own event too, before the confirmation:
> w006deploy-42
< a006deploy-42
< v0062/2/0
The confirmation is received/sent/errors:
received: how many clients confirmed the event.sent: how many clients it was dispatched to. On TCP this is every connection open at that moment, including the writer.errors: deliveries that failed, or that weren't confirmed within 200 ms.received + errorsalways equalssent.
Ghoti only answers the write once every delivery is confirmed or the 200 ms wait is over, so many clients, or slow ones, delay the confirmation.
- At most once. One attempt per client and no retries. A client that is slow, disconnecting or whose queue is full misses the event. Like Redis Pub/Sub or Core NATS, it is not a durable queue.
- A read returns the last value written.
- Every write sends the event again, so a write is not safe to retry.
- Events reach every connection, whatever the slot's read permissions are. The
usersmap controls who can read and write the slot, not who receives its events. - Over the
httptransport, aGETon a broadcast slot opens a Server-Sent Events stream, and events go to the open streams only.
Multicast#
kind: multicast. Like broadcast, but the event only goes to the clients that joined the slot's group with s. A client leaves the group with d.
| Option | Required | Description |
|---|---|---|
timeout | No, default 200 | Milliseconds to wait for subscribers to confirm an event. At least 1. Note that this is milliseconds, unlike timeout_memory's seconds. |
dereg_tries | No, default 3 | Consecutive failed deliveries after which a subscriber is removed from the group. At least 1. |
slot_007:
kind: multicast
timeout: 200
dereg_tries: 3
Captured with a subscriber X and a writer B:
# X joins
> s007
< v0071
# B writes; only X receives the event
> w007cache-flush
< v0071/1/0
# B joins too, then writes again
> s007
< v0071
> w007hi
< a007hi
< v0072/2/0
sanswers1anddanswers0: whether the connection is a member now. Both are idempotent and need read permission.- A write answers
received/sent/errors, counting group members only. With no members, the answer is0/0/0. - A member that fails
dereg_triesdeliveries in a row, for example because it disconnected withoutd, is removed from the group. - Delivery is at most once, with no retries, exactly as for broadcast.
- A read returns the last value written.
- Groups can't be joined over the
httptransport.
Planned: random signal propagation#
Planned A slot kind that delivers each write to one subscriber chosen at random, instead of to all of them. It is not implemented, and a slot configured with an unknown kind is ignored.