Operations
What you need to run Ghoti somewhere other than your laptop: metrics, connection limits, process lifecycle and TLS.
Prometheus metrics#
Ghoti can expose runtime metrics for Prometheus. When enabled, a dedicated HTTP listener serves GET /metrics in the Prometheus text exposition format. It is disabled by default.
metrics:
enabled: true
addr: "127.0.0.1:9100" # host:port where GET /metrics is served
| Key | Description |
|---|---|
metrics.enabled | Enables metrics collection and the /metrics endpoint. Default false. |
metrics.addr | host:port of the metrics listener. Required when enabled. |
The endpoint is unauthenticated, so bind it to a private interface or firewall it as you would any Prometheus target. Scrape it with a standard job:
scrape_configs:
- job_name: ghoti
static_configs:
- targets: ["127.0.0.1:9100"]
Besides the standard Go runtime (go_*) and process (process_*) collectors, Ghoti exposes:
| Metric | Type | Description |
|---|---|---|
ghoti_connected_clients | gauge | Number of currently connected clients. |
ghoti_requests_total | counter | Total number of requests processed. Use rate() for requests per second. |
ghoti_request_duration_seconds | histogram | Request processing time in seconds, with buckets from 50 µs to 1 s. Use histogram_quantile() for tail latency. |
For example, requests per second and p99 processing time over the last five minutes:
rate(ghoti_requests_total[5m])
histogram_quantile(0.99, rate(ghoti_request_duration_seconds_bucket[5m]))
Before version 0.2, metrics were written to files. The old keys (output_dir, rotation, retain, interval) are now rejected at startup so the change isn't missed.
Connection limits#
By default Ghoti accepts as many client connections as the operating system allows. max_connections caps how many can be open at once, on every transport:
max_connections: 1000
- When the cap is reached, new connections are not refused. They complete the TCP handshake and wait in the kernel's accept queue until a connection closes, so a client sees a connect that succeeds and a first response that is delayed.
- Over
httpthe cap also counts SSE subscribers, because each stream holds a connection for as long as it is open. 0, the default, means no cap. A negative value stops the server at startup.
Deployment#
- One process, one binary. Download a release archive, put
config.yamlin/etc/ghoti, and startghoti rununder your process supervisor. Logs go to standard output, intextorjsonformat. - Bind deliberately. The default
addrislocalhost:9090, reachable from the same host only. Prefer a specific private interface over0.0.0.0when other hosts need access, and put TLS in front of it (below). - Plan for restarts. A restart empties every slot, drops every subscription and forgets every
timeout_memoryowner. Clients must reconnect, log in again and subscribe again. - Shutdown.
SIGINTorSIGTERMstops the server. It logsShutting down serverand closes the listeners and connections. - Single point of failure. A standalone server is one. The experimental cluster mode adds failover, but the new leader starts empty.
When ghoti run fails to start, the exit status tells you which step failed:
| Exit status | Cause |
|---|---|
1 | The configuration couldn't be loaded or is invalid. |
3 | The cluster couldn't be created: an unsupported cluster.manager.type, or cluster.user or cluster.pass shorter than 4 characters. A failed join does not stop the server; see Cluster. |
4 | The server couldn't start listening on addr. |
5 | The metrics listener couldn't start. |
TLS#
Ghoti has no TLS on any listener: not the protocol port, not the HTTP and SSE transport, not the cluster manager, and not the metrics endpoint. On these connections, the p command, the HTTP Basic Auth header and the cluster credentials all travel in cleartext.
For any deployment that isn't entirely local:
- Terminate TLS in external infrastructure, such as a load balancer, a reverse proxy or a service mesh sidecar, in front of every listener, and keep the plaintext hop between the proxy and Ghoti on loopback.
- For cluster traffic between nodes, use mutual TLS, a VPN or another encrypted transport. The cluster credentials are shared by every node, so a single observed request compromises the whole cluster.
- Otherwise keep all traffic on a trusted network, and never expose a plaintext listener to an untrusted one.
The full security model, and how to report a vulnerability, are in SECURITY.md.