Benchmarks · Ghoti v0.3.0

Sub-millisecond answers from a 2-core, 1 GiB server.

Ghoti ran alone on an AWS t4g.micro, a small burstable Graviton instance, and a separate, bigger machine sent the load over the network. Every request was checked, and every number below can be reproduced with ghoti-bench.

0.29 ms

Median round trip with one connection, network included

memory-read, p50. p99.9 under 0.75 ms in every tested scenario.

134,343 req/s

Memory reads with 100 connections, on 2 cores

Median of 3 runs of 5 minutes, with a 0.60 ms median round trip.

7.6 µs

Average time inside Ghoti per memory read

Over 247,423,099 requests; 99.83% took under 50 µs.

0 errors

Over 255,244,437 requests in 21 runs

Every value, counter and token bucket gave the answer it had to.

Setup

A small server, a big load generator.

Both instances run in the same subnet of one availability zone in AWS us-west-2. The generator is much bigger than the server, so the numbers measure Ghoti, not the benchmark.

Load generator

c7g.xlarge

4 Graviton3 cores, running ghoti-bench. A run only counts if the generator stayed under 80% CPU.

Server

t4g.micro

2 Graviton2 cores, 1 GiB of memory, unlimited CPU credits. The released v0.3.0 linux/arm64 binary, checked against the release checksums.

  • One request in flight per connection. Each connection sends a request, waits for the whole answer, checks it, and only then sends the next. That's the only way Ghoti supports.
  • All connections opened first and kept open for the whole run. A warm-up runs before every measurement and isn't counted.
  • Two clocks. The generator times every round trip, network included. Ghoti's own Prometheus histogram times each request inside the server.
  • Strict validity. A run counts only if no connection dropped, there were no errors, and the correctness checks passed.

Latency with one connection

The fastest a client can get an answer.

Measured on the client, so each time includes the network round trip between the two instances. Every tested scenario answers in about the same time, including atomic counters and token buckets that every connection shares.

Median round trip (p50)

1 connection, milliseconds

memory-read0.290
counter-hot0.282
counter-spread0.294
token-bucket-hot0.242
token-bucket-spread0.242

Slowest 1 in 1,000 (p99.9)

1 connection, milliseconds

memory-read0.597
counter-hot0.747
counter-spread0.552
token-bucket-hot0.681
token-bucket-spread0.719
Scenariop50 msp90 msp99 msp99.9 ms
memory-read0.2900.3220.3720.597
counter-hot0.2820.3090.3680.747
counter-spread0.2940.3210.3690.552
token-bucket-hot0.2420.2740.3320.681
token-bucket-spread0.2420.2860.3680.719

memory-read reads a 36-byte value from a simple memory slot.

counter-hot increments one atomic counter shared by every connection; counter-spread one of 100, chosen at random.

token-bucket-hot takes a token from one shared token bucket refilled at 1,000 per second; token-bucket-spread from one of 100 buckets at 10 per second each.

Latency under load

Over 127,000 requests per second, still under a millisecond at the median.

With 100 connections sending at full speed, every tested scenario served between 127,594 and 139,552 requests per second on the 2-core server, and half of the requests came back in about 0.6 ms. The slowest requests take a few milliseconds, because some requests wait for others to be answered first.

Median round trip (p50)

100 connections, milliseconds

memory-read0.600
counter-hot0.628
counter-spread0.602
token-bucket-hot0.588
token-bucket-spread0.591

Requests per second

100 connections

memory-read134,343
counter-hot127,594
counter-spread134,746
token-bucket-hot139,552
token-bucket-spread139,188
ScenarioRequests/sp50 msp90 msp99 msp99.9 msServer CPU
memory-read134,3430.6001.1013.5417.12382%
counter-hot127,5940.6281.1684.0538.86478%
counter-spread134,7460.6021.0923.4947.88183%
token-bucket-hot139,5520.5881.0753.3797.80386%
token-bucket-spread139,1880.5911.0813.2387.52086%

memory-read is the median of three runs; the other slot types are one run each, so differences of a few percent between them are within what one run can show.


Time inside Ghoti

A few microseconds per request.

Ghoti's own metrics time each request from the moment the complete request arrives until the answer is written to the connection. Almost all of a sub-millisecond round trip is the network, the operating systems and the client.

These cover every request of each session, warm-ups and all connection counts included. The histogram's smallest bucket is 50 µs, so the shares below are bucket boundaries.

SessionRequestsAverageUnder 50 µsUnder 1 ms
Memory reads247,423,0997.6 µs99.83%99.97%
Counters17,275,41311.6 µs99.64%99.91%
Token buckets17,976,49011.2 µs99.65%99.92%

1,000 connections

Throughput holds; requests wait their turn.

With 1,000 connections the server still served over 110,000 requests per second, using 88 to 90% of its 2 cores. The median round trip rises to about 8.4 ms. Ghoti isn't slower per request; 1,000 clients that each keep one request waiting share the same rate, so each waits about 1,000 / 112,647 ≈ 8.9 ms for its turn, which matches the measured average.

Requests per second

1,000 connections

memory-read112,647
counter-hot110,026
counter-spread112,955
token-bucket-hot113,173
token-bucket-spread113,062

Median round trip (p50)

1,000 connections, milliseconds

memory-read8.376
counter-hot8.552
counter-spread8.385
token-bucket-hot8.339
token-bucket-spread8.397

Correctness

Fast only counts if it's right.

Every memory read was compared byte by byte with the value the slot holds. Counters and token buckets change on every request, so they were checked after each run.

Atomic counters

Read back after each run, every counter held exactly the number of increments the benchmark sent: nothing lost, nothing applied twice. That held in all 6 runs, up to 4,716,734 increments with 100 connections.

Token buckets

Under constant load, each bucket handed out every token it refilled and never more than it can hold plus its refills. For example, the shared bucket at 1,000 tokens per second gave out exactly 30,000 tokens in the 30-second window with 100 and 1,000 connections.

Zero errors

All 21 runs finished with 0 errors over 255,244,437 requests, and every connection stayed open for the whole run.

Reading these numbers

What they do and don't tell you.

  • Your network decides the round trip. Both instances sat in one availability zone, the shortest network EC2 offers. Expect a different round trip from yours. The time inside Ghoti doesn't depend on the network.
  • This is version 0.3.0 on one instance type, with the plain-text standard transport over TCP and no TLS. The http transport, broadcast and multicast slots, and clustering weren't measured.
  • One run for most scenarios. Only memory-read has three repetitions; counters and token buckets ran once for 30 seconds each.
  • Maximums are higher. The slowest single round trip of any run ranged from 5.7 ms with one connection to 255 ms with 1,000. They are in the full table.

Reproduce it.

With AWS credentials, Go, ssh and curl, one script creates both instances, runs the benchmark, downloads the results and deletes everything it created, even when it fails. The memory session takes about an hour.

$ git clone https://github.com/dankomiocevic/ghoti-bench.git && cd ghoti-bench
$ AWS_REGION=us-west-2 GHOTI_REF=v0.3.0 scripts/ec2-bench.sh

All results

Every case, in one table.

Ghoti v0.3.0 on a t4g.micro: the median of the valid runs for every case. Max is the slowest round trip of any run.

ScenarioConnsRunsRequests/sp50 msp90 msp99 msp99.9 msmax msCPUErrors
memory-read133,3590.2900.3220.3720.59716.8296%0
memory-read1003134,3430.6001.1013.5417.12357.18582%0
memory-read1,0003112,6478.37611.19018.69426.231255.35689%0
counter-hot113,4530.2820.3090.3680.7479.7546%0
counter-hot1001127,5940.6281.1684.0538.86474.56678%0
counter-hot1,0001110,0268.55211.50219.30036.07871.77988%0
counter-spread113,3200.2940.3210.3690.5525.6976%0
counter-spread1001134,7460.6021.0923.4947.88151.75283%0
counter-spread1,0001112,9558.38511.13317.99024.78932.15190%0
token-bucket-hot113,9960.2420.2740.3320.68110.7647%0
token-bucket-hot1001139,5520.5881.0753.3797.80365.05986%0
token-bucket-hot1,0001113,1738.33911.17418.71126.329223.58389%0
token-bucket-spread113,9570.2420.2860.3680.7198.9527%0
token-bucket-spread1001139,1880.5911.0813.2387.52027.37886%0
token-bucket-spread1,0001113,0628.39711.03517.84225.068222.23790%0

Source: ghoti-bench RESULTS.md, Ghoti v0.3.0 benchmark report of 2026-11-19.