ClickHouse Launches PostgresBench: An Open, Reproducible Benchmark for Managed Postgres Services
ClickHouse has open-sourced PostgresBench, a benchmark framework for comparing managed PostgreSQL services. The benchmark uses the standard pgbench tool with a TPC-B-like workload, testing five services at two data sizes (100 GB and 500 GB). All results, configurations, and scripts are public on GitHub.
Why a New Benchmark?
ClickHouse's managed Postgres service is new. To prove its performance claims, the team wanted a transparent methodology—just like they did with ClickBench for OLAP databases. The goal: any developer can reproduce results, submit improvements, or flag issues.
Benchmark Design
PostgresBench runs pgbench with 256 concurrent clients and 16 threads for 10 minutes. The command:
pgbench -c 256 -j 16 -T 600 -M prepared -P 30 \
-s $SCALE_FACTOR \
-h $PGHOST -p $PGPORT -U $PGUSER -d $PGDATABASE
Two scale factors: 6849 (~100 GB) and 34247 (~500 GB). These represent small and medium deployments where the working set fits in cache or starts spilling to disk.
Metrics reported: average TPS, average latency, P95, and P99 latency across three runs per configuration.
Tested Services and Configurations
- Postgres managed by ClickHouse (m8gd.xlarge: 4 vCPU/16 GB NVMe; m8gd.4xlarge: 16 vCPU/64 GB NVMe)
- AWS Aurora PostgreSQL (db.r6gd.xlarge: 4 vCPU/32 GB NVMe; db.r6g.4xlarge: 16 vCPU/128 GB)
- AWS RDS for PostgreSQL (db.m8gd.xlarge: 4 vCPU/16 GB NVMe + 1000 GB GP3; db.m8gd.4xlarge: 16 vCPU/64 GB)
- Neon (Serverless: 4 vCPU/16 GB and 16 vCPU/64 GB)
- Crunchy Bridge (Standard-16: 4 vCPU/16 GB with 6K baseline IOPS; Standard-64: 16 vCPU/64 GB with 20K IOPS)
All tests ran in us-east-2 with HA disabled, using Postgres 17 or 18 (Aurora on 17).
Results at 100 GB (Scale Factor 6849)
| Service | TPS | Avg Latency (ms) | P95 (ms) | P99 (ms) |
|---|---|---|---|---|
| ClickHouse Small | 6172 | 41.46 | 64.02 | 80.89 |
| ClickHouse Large | 28668 | 8.90 | 10.23 | 11.68 |
| Aurora Small | 2685 | 95.29 | 165.54 | 298.39 |
| Aurora Large | 12628 | 20.24 | 30.97 | 39.04 |
| RDS Small | 4882 | 52.41 | 98.00 | 124.19 |
| RDS Large | 8133 | 31.43 | 72.50 | 97.68 |
| Neon Small | 2847 | 89.90 | 106.14 | 116.47 |
| Neon Large | 8563 | 29.83 | 41.42 | 49.21 |
| Crunchy Small | 6338 | 40.37 | 66.10 | 85.83 |
| Crunchy Large | 14790 | 17.26 | 28.32 | 34.61 |
Results at 500 GB (Scale Factor 34247)
| Service | TPS | Avg Latency (ms) | P95 (ms) | P99 (ms) |
|---|---|---|---|---|
| ClickHouse Large | 26328 | 9.70 | 11.40 | 13.19 |
| Aurora Large | 10402 | 24.58 | 36.17 | 46.49 |
| RDS Large | 5092 | 50.23 | 88.65 | 117.90 |
| Neon Large | 7802 | 32.80 | 47.53 | 56.30 |
| Crunchy Large | 11113 | 22.99 | 36.40 | 41.68 |
Why NVMe Matters
The TPC-B workload is write-heavy due to continuous UPDATEs generating WAL records. ClickHouse's NVMe storage is physically co-located with compute, delivering microsecond disk latency. In contrast, EBS-backed volumes add network latency to every fsync. The article states: "Most of the time, Postgres isn’t slow, your storage is."
How to Reproduce
Clone the repository and run:
export PGHOST="your-database-host"
export PGPORT=5432
export PGUSER="postgres"
export PGPASSWORD="your-password"
export PGDATABASE="postgres"
export VCPUS=16
export RAM_GB=64
export SYSTEM_NAME="Postgres by ClickHouse"
export INSTANCE_TYPE="m8gd.4xlarge"
export INSTANCE_STORAGE="950 GB - NVMe"
export PRIMARY_STORAGE="NVMe"
export OUT_JSON="results.json"
./run.sh
The script initializes data, runs the benchmark three times, and outputs JSON. Submit results via PR.
What's Next
- Pricing comparisons are not included (ClickHouse's service wasn't public at test time).
- Future plans: HA configurations, custom Postgres configs, and colocation by availability zone.
- Contributions welcome to expand the benchmark.
