Concepts
Four ideas explain almost every behaviour in Ghoti: slots, volatile state, connections as identities, and one request in flight per connection.
Slots#
A Ghoti server has 1000 slots, numbered 000 to 999. A slot exists only if the configuration defines it, as slot_000 to slot_999 with a kind. A command that addresses an undefined slot gets error 005 (MISSING_SLOT).
The kind decides what reads and writes do. On a simple_memory slot a read is free of side effects. On atomic, token_bucket, leaky_bucket and ticker slots every read changes the state. On broadcast and multicast slots every write sends events to other clients. The slot reference covers each kind.
Clients must know the configuration beforehand: the protocol has no command to list slots or ask for a slot's kind. If a slot's configuration changes, Ghoti can't keep the data consistent until every server runs the new configuration.
Values are opaque byte strings of at most 36 bytes. Ghoti never validates, normalises or transcodes them, so binary data is allowed, and a read returns exactly the bytes that were written.
If a slot's definition is invalid, for example an unknown kind or a ticker without refresh_rate, the server still starts and logs nothing about it. That slot behaves as if it was never defined and answers e<slot>005. After changing the configuration, read each slot once to confirm it exists.
Volatile state#
Ghoti keeps everything in memory and writes nothing to disk. When the process stops, every value, counter, bucket, subscription and lease owner is gone. When it starts again, every slot is back at its initial state.
The same applies to a cluster: clustering elects a leader but doesn't replicate data, so a newly elected leader starts empty.
Design the system so that losing Ghoti's state costs performance or a short degradation, never correctness. If a slot is a cache, the system must still work when it is empty. If it holds a lease, the system must tolerate two holders around a restart.
Connections and ownership#
On the standard and telnet transports, a client is a TCP connection. Ghoti tracks two things per connection:
- The logged-in user, set by the
uandpcommands. See Authentication. - Identity for ownership and subscriptions. A
timeout_memoryslot is owned by the connection that wrote it, and amulticastsubscription belongs to the connection that sents.
Ownership is tied to the connection, not to the user or the host. Two connections from the same process, or logged in as the same user, are two different owners. This has consequences:
- A dropped connection does not release a lease. The owner stays the owner until its timeout expires. A client that reconnects right after a drop is a new connection, so its writes fail with
007until the old timeout runs out. - A restart forgets the owner. After Ghoti restarts the slot has no owner, and the first connection to write takes it, even if a client still believes it holds the lease.
- Over HTTP every request is its own connection. No HTTP client can own a
timeout_memoryslot across two writes, and HTTP clients cannot subscribe to multicast slots.
One request in flight#
A client must send a command and wait for its response before it sends the next one. Ghoti handles a connection's commands one at a time and finishes writing each response before it reads the next command. That gives a strict one-request, one-response relation, and it is the only supported way to use a connection. Pipelining (sending more commands before the response arrives) is not supported, and Ghoti may reject it in the future.
To run commands in parallel, open more connections. Keep in mind that leases and subscriptions belong to the connection that created them.
Async events#
Besides answers to its own commands, a connection can receive async events: lines that start with a, produced by another client's write to a broadcast slot or to a multicast slot this connection subscribed to. They can arrive at any moment, before or after any response but never in the middle of one, so a client has to split the stream on \n and route lines by their first byte. The protocol page has the exact rules.