add rely-sqlite to benchmark
Some checks failed
Go / build-and-release (push) Has been cancelled

This commit is contained in:
2025-11-20 20:55:37 +00:00
parent 00a6a78a41
commit 55add34ac1
54 changed files with 5519 additions and 4442 deletions

View File

@@ -0,0 +1,194 @@
================================================================
NOSTR RELAY BENCHMARK AGGREGATE REPORT
================================================================
Generated: 2025-11-20T20:32:25+00:00
Benchmark Configuration:
Events per test: 50000
Concurrent workers: 24
Test duration: 60s
Relays tested: 9
================================================================
SUMMARY BY RELAY
================================================================
Relay: rely-sqlite
----------------------------------------
Status: COMPLETED
Events/sec: 17507.10
Events/sec: 6243.12
Events/sec: 17507.10
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.223582ms
Bottom 10% Avg Latency: 698.877µs
Avg Latency: 1.178662ms
P95 Latency: 1.87223ms
P95 Latency: 2.046981ms
P95 Latency: 883.507µs
Relay: next-orly-badger
----------------------------------------
Status: COMPLETED
Events/sec: 16840.34
Events/sec: 6128.23
Events/sec: 16840.34
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.297596ms
Bottom 10% Avg Latency: 722.094µs
Avg Latency: 1.265918ms
P95 Latency: 2.027536ms
P95 Latency: 2.302166ms
P95 Latency: 894.834µs
Relay: next-orly-dgraph
----------------------------------------
Status: COMPLETED
Events/sec: 16563.45
Events/sec: 6132.86
Events/sec: 16563.45
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.32589ms
Bottom 10% Avg Latency: 726.176µs
Avg Latency: 1.340819ms
P95 Latency: 2.152481ms
P95 Latency: 2.37338ms
P95 Latency: 904.165µs
Relay: next-orly-neo4j
----------------------------------------
Status: COMPLETED
Events/sec: 14622.22
Events/sec: 6182.48
Events/sec: 14622.22
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.559545ms
Bottom 10% Avg Latency: 795.698µs
Avg Latency: 1.269605ms
P95 Latency: 2.658118ms
P95 Latency: 2.293256ms
P95 Latency: 867.888µs
Relay: khatru-sqlite
----------------------------------------
Status: COMPLETED
Events/sec: 16872.81
Events/sec: 6219.91
Events/sec: 16872.81
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.294206ms
Bottom 10% Avg Latency: 724.237µs
Avg Latency: 1.28288ms
P95 Latency: 2.011193ms
P95 Latency: 2.16732ms
P95 Latency: 868.521µs
Relay: khatru-badger
----------------------------------------
Status: COMPLETED
Events/sec: 15204.92
Events/sec: 6277.98
Events/sec: 15204.92
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.485679ms
Bottom 10% Avg Latency: 768.979µs
Avg Latency: 1.216531ms
P95 Latency: 2.501619ms
P95 Latency: 2.028348ms
P95 Latency: 862.271µs
Relay: relayer-basic
----------------------------------------
Status: COMPLETED
Events/sec: 17272.97
Events/sec: 6207.90
Events/sec: 17272.97
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.255956ms
Bottom 10% Avg Latency: 712.498µs
Avg Latency: 1.21703ms
P95 Latency: 1.909735ms
P95 Latency: 2.233521ms
P95 Latency: 871.278µs
Relay: strfry
----------------------------------------
Status: COMPLETED
Events/sec: 15745.79
Events/sec: 6264.53
Events/sec: 15745.79
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.415908ms
Bottom 10% Avg Latency: 739.523µs
Avg Latency: 1.153768ms
P95 Latency: 2.340716ms
P95 Latency: 2.007502ms
P95 Latency: 855.87µs
Relay: nostr-rs-relay
----------------------------------------
Status: COMPLETED
Events/sec: 17638.66
Events/sec: 6241.74
Events/sec: 17638.66
Success Rate: 100.0%
Success Rate: 100.0%
Success Rate: 100.0%
Avg Latency: 1.18563ms
Bottom 10% Avg Latency: 646.954µs
Avg Latency: 1.182584ms
P95 Latency: 1.847889ms
P95 Latency: 2.120267ms
P95 Latency: 866.51µs
================================================================
DETAILED RESULTS
================================================================
Individual relay reports are available in:
- /reports/run_20251120_200202/khatru-badger_results.txt
- /reports/run_20251120_200202/khatru-sqlite_results.txt
- /reports/run_20251120_200202/next-orly-badger_results.txt
- /reports/run_20251120_200202/next-orly-dgraph_results.txt
- /reports/run_20251120_200202/next-orly-neo4j_results.txt
- /reports/run_20251120_200202/nostr-rs-relay_results.txt
- /reports/run_20251120_200202/relayer-basic_results.txt
- /reports/run_20251120_200202/rely-sqlite_results.txt
- /reports/run_20251120_200202/strfry_results.txt
================================================================
BENCHMARK COMPARISON TABLE
================================================================
Relay Status Peak Tput/s Avg Latency Success Rate
---- ------ ----------- ----------- ------------
rely-sqlite OK 17507.10 1.223582ms 100.0%
next-orly-badger OK 16840.34 1.297596ms 100.0%
next-orly-dgraph OK 16563.45 1.32589ms 100.0%
next-orly-neo4j OK 14622.22 1.559545ms 100.0%
khatru-sqlite OK 16872.81 1.294206ms 100.0%
khatru-badger OK 15204.92 1.485679ms 100.0%
relayer-basic OK 17272.97 1.255956ms 100.0%
strfry OK 15745.79 1.415908ms 100.0%
nostr-rs-relay OK 17638.66 1.18563ms 100.0%
================================================================
End of Report
================================================================

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_khatru-badger_8
Events: 50000, Workers: 24, Duration: 1m0s
1763669935332908 migrating to version 1... /build/pkg/database/migrations.go:66
1763669935332973 migrating to version 2... /build/pkg/database/migrations.go:73
1763669935332998 migrating to version 3... /build/pkg/database/migrations.go:80
1763669935333005 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763669935333040 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763669935333094 migrating to version 4... /build/pkg/database/migrations.go:87
1763669935333104 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763669935333122 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763669935333128 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:18:55 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:18:55 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 3.288409868s
Events/sec: 15204.92
Avg latency: 1.485679ms
P90 latency: 2.12405ms
P95 latency: 2.501619ms
P99 latency: 3.714496ms
Bottom 10% Avg latency: 768.979µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 324.753031ms
Burst completed: 5000 events in 291.367672ms
Burst completed: 5000 events in 301.649121ms
Burst completed: 5000 events in 328.41364ms
Burst completed: 5000 events in 281.252591ms
Burst completed: 5000 events in 328.008049ms
Burst completed: 5000 events in 310.281138ms
Burst completed: 5000 events in 260.825936ms
Burst completed: 5000 events in 270.80417ms
Burst completed: 5000 events in 258.334978ms
Burst test completed: 50000 events in 7.964347994s, errors: 0
Events/sec: 6277.98
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.423948265s
Combined ops/sec: 2047.17
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 415254 queries in 1m0.003601442s
Queries/sec: 6920.48
Avg query latency: 1.603002ms
P95 query latency: 6.256605ms
P99 query latency: 9.899737ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 325890 operations (275890 queries, 50000 writes) in 1m0.003099307s
Operations/sec: 5431.22
Avg latency: 1.378137ms
Avg query latency: 1.366065ms
Avg write latency: 1.44475ms
P95 latency: 3.427873ms
P99 latency: 5.340723ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 3.288409868s
Total Events: 50000
Events/sec: 15204.92
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 126 MB
Avg Latency: 1.485679ms
P90 Latency: 2.12405ms
P95 Latency: 2.501619ms
P99 Latency: 3.714496ms
Bottom 10% Avg Latency: 768.979µs
----------------------------------------
Test: Burst Pattern
Duration: 7.964347994s
Total Events: 50000
Events/sec: 6277.98
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 161 MB
Avg Latency: 1.216531ms
P90 Latency: 1.748877ms
P95 Latency: 2.028348ms
P99 Latency: 2.847978ms
Bottom 10% Avg Latency: 540.737µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.423948265s
Total Events: 50000
Events/sec: 2047.17
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 132 MB
Avg Latency: 369.523µs
P90 Latency: 775.926µs
P95 Latency: 862.271µs
P99 Latency: 1.05139ms
Bottom 10% Avg Latency: 976.651µs
----------------------------------------
Test: Query Performance
Duration: 1m0.003601442s
Total Events: 415254
Events/sec: 6920.48
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 118 MB
Avg Latency: 1.603002ms
P90 Latency: 4.760818ms
P95 Latency: 6.256605ms
P99 Latency: 9.899737ms
Bottom 10% Avg Latency: 6.959951ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003099307s
Total Events: 325890
Events/sec: 5431.22
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 139 MB
Avg Latency: 1.378137ms
P90 Latency: 2.762527ms
P95 Latency: 3.427873ms
P99 Latency: 5.340723ms
Bottom 10% Avg Latency: 3.863556ms
----------------------------------------
Report saved to: /tmp/benchmark_khatru-badger_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_khatru-badger_8/benchmark_report.adoc
RELAY_NAME: khatru-badger
RELAY_URL: ws://khatru-badger:3334
TEST_TIMESTAMP: 2025-11-20T20:22:13+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_khatru-sqlite_8
Events: 50000, Workers: 24, Duration: 1m0s
1763669732839163 migrating to version 1... /build/pkg/database/migrations.go:66
1763669732839345 migrating to version 2... /build/pkg/database/migrations.go:73
1763669732839423 migrating to version 3... /build/pkg/database/migrations.go:80
1763669732839433 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763669732839447 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763669732839469 migrating to version 4... /build/pkg/database/migrations.go:87
1763669732839476 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763669732839496 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763669732839504 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:15:32 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:15:32 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 2.963346692s
Events/sec: 16872.81
Avg latency: 1.294206ms
P90 latency: 1.715271ms
P95 latency: 2.011193ms
P99 latency: 3.190375ms
Bottom 10% Avg latency: 724.237µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 291.855294ms
Burst completed: 5000 events in 316.021528ms
Burst completed: 5000 events in 282.131412ms
Burst completed: 5000 events in 299.105944ms
Burst completed: 5000 events in 267.419607ms
Burst completed: 5000 events in 325.020614ms
Burst completed: 5000 events in 305.340591ms
Burst completed: 5000 events in 271.0695ms
Burst completed: 5000 events in 390.24426ms
Burst completed: 5000 events in 284.381622ms
Burst test completed: 50000 events in 8.038707278s, errors: 0
Events/sec: 6219.91
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.489286115s
Combined ops/sec: 2041.71
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 420505 queries in 1m0.003538635s
Queries/sec: 7008.00
Avg query latency: 1.572366ms
P95 query latency: 6.018765ms
P99 query latency: 9.565009ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 323946 operations (273946 queries, 50000 writes) in 1m0.003027777s
Operations/sec: 5398.83
Avg latency: 1.414998ms
Avg query latency: 1.390113ms
Avg write latency: 1.551346ms
P95 latency: 3.512421ms
P99 latency: 5.637893ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 2.963346692s
Total Events: 50000
Events/sec: 16872.81
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 205 MB
Avg Latency: 1.294206ms
P90 Latency: 1.715271ms
P95 Latency: 2.011193ms
P99 Latency: 3.190375ms
Bottom 10% Avg Latency: 724.237µs
----------------------------------------
Test: Burst Pattern
Duration: 8.038707278s
Total Events: 50000
Events/sec: 6219.91
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 208 MB
Avg Latency: 1.28288ms
P90 Latency: 1.849315ms
P95 Latency: 2.16732ms
P99 Latency: 3.046622ms
Bottom 10% Avg Latency: 581.238µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.489286115s
Total Events: 50000
Events/sec: 2041.71
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 199 MB
Avg Latency: 372.036µs
P90 Latency: 778.229µs
P95 Latency: 868.521µs
P99 Latency: 1.078812ms
Bottom 10% Avg Latency: 1.036235ms
----------------------------------------
Test: Query Performance
Duration: 1m0.003538635s
Total Events: 420505
Events/sec: 7008.00
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 124 MB
Avg Latency: 1.572366ms
P90 Latency: 4.639693ms
P95 Latency: 6.018765ms
P99 Latency: 9.565009ms
Bottom 10% Avg Latency: 6.728349ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003027777s
Total Events: 323946
Events/sec: 5398.83
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 104 MB
Avg Latency: 1.414998ms
P90 Latency: 2.807811ms
P95 Latency: 3.512421ms
P99 Latency: 5.637893ms
Bottom 10% Avg Latency: 4.028549ms
----------------------------------------
Report saved to: /tmp/benchmark_khatru-sqlite_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_khatru-sqlite_8/benchmark_report.adoc
RELAY_NAME: khatru-sqlite
RELAY_URL: ws://khatru-sqlite:3334
TEST_TIMESTAMP: 2025-11-20T20:18:50+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_next-orly-badger_8
Events: 50000, Workers: 24, Duration: 1m0s
1763669124600787 migrating to version 1... /build/pkg/database/migrations.go:66
1763669124600839 migrating to version 2... /build/pkg/database/migrations.go:73
1763669124600865 migrating to version 3... /build/pkg/database/migrations.go:80
1763669124600871 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763669124600882 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763669124600896 migrating to version 4... /build/pkg/database/migrations.go:87
1763669124600900 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763669124600913 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763669124600919 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:05:24 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:05:24 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 2.969061628s
Events/sec: 16840.34
Avg latency: 1.297596ms
P90 latency: 1.734511ms
P95 latency: 2.027536ms
P99 latency: 2.961433ms
Bottom 10% Avg latency: 722.094µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 276.383103ms
Burst completed: 5000 events in 347.587541ms
Burst completed: 5000 events in 381.7012ms
Burst completed: 5000 events in 339.439731ms
Burst completed: 5000 events in 292.19598ms
Burst completed: 5000 events in 338.289935ms
Burst completed: 5000 events in 335.224221ms
Burst completed: 5000 events in 271.373815ms
Burst completed: 5000 events in 290.588853ms
Burst completed: 5000 events in 278.611302ms
Burst test completed: 50000 events in 8.15896297s, errors: 0
Events/sec: 6128.23
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.531766787s
Combined ops/sec: 2038.17
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 406469 queries in 1m0.004230933s
Queries/sec: 6774.01
Avg query latency: 1.643787ms
P95 query latency: 6.491386ms
P99 query latency: 10.300562ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 321891 operations (271891 queries, 50000 writes) in 1m0.003425476s
Operations/sec: 5364.54
Avg latency: 1.412817ms
Avg query latency: 1.395014ms
Avg write latency: 1.509627ms
P95 latency: 3.531794ms
P99 latency: 5.566648ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 2.969061628s
Total Events: 50000
Events/sec: 16840.34
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 163 MB
Avg Latency: 1.297596ms
P90 Latency: 1.734511ms
P95 Latency: 2.027536ms
P99 Latency: 2.961433ms
Bottom 10% Avg Latency: 722.094µs
----------------------------------------
Test: Burst Pattern
Duration: 8.15896297s
Total Events: 50000
Events/sec: 6128.23
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 204 MB
Avg Latency: 1.265918ms
P90 Latency: 1.967513ms
P95 Latency: 2.302166ms
P99 Latency: 3.178464ms
Bottom 10% Avg Latency: 442.546µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.531766787s
Total Events: 50000
Events/sec: 2038.17
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 140 MB
Avg Latency: 385.858µs
P90 Latency: 804.273µs
P95 Latency: 894.834µs
P99 Latency: 1.119529ms
Bottom 10% Avg Latency: 1.040121ms
----------------------------------------
Test: Query Performance
Duration: 1m0.004230933s
Total Events: 406469
Events/sec: 6774.01
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 120 MB
Avg Latency: 1.643787ms
P90 Latency: 4.902634ms
P95 Latency: 6.491386ms
P99 Latency: 10.300562ms
Bottom 10% Avg Latency: 7.252457ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003425476s
Total Events: 321891
Events/sec: 5364.54
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 120 MB
Avg Latency: 1.412817ms
P90 Latency: 2.823412ms
P95 Latency: 3.531794ms
P99 Latency: 5.566648ms
Bottom 10% Avg Latency: 4.024306ms
----------------------------------------
Report saved to: /tmp/benchmark_next-orly-badger_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_next-orly-badger_8/benchmark_report.adoc
RELAY_NAME: next-orly-badger
RELAY_URL: ws://next-orly-badger:8080
TEST_TIMESTAMP: 2025-11-20T20:08:42+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_next-orly-dgraph_8
Events: 50000, Workers: 24, Duration: 1m0s
1763669327215819 migrating to version 1... /build/pkg/database/migrations.go:66
1763669327215873 migrating to version 2... /build/pkg/database/migrations.go:73
1763669327215897 migrating to version 3... /build/pkg/database/migrations.go:80
1763669327215903 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763669327215913 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763669327215942 migrating to version 4... /build/pkg/database/migrations.go:87
1763669327215950 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763669327215962 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763669327215968 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:08:47 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:08:47 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 3.018694521s
Events/sec: 16563.45
Avg latency: 1.32589ms
P90 latency: 1.831543ms
P95 latency: 2.152481ms
P99 latency: 3.113153ms
Bottom 10% Avg latency: 726.176µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 292.171946ms
Burst completed: 5000 events in 318.508865ms
Burst completed: 5000 events in 366.003137ms
Burst completed: 5000 events in 299.686978ms
Burst completed: 5000 events in 285.823742ms
Burst completed: 5000 events in 329.930802ms
Burst completed: 5000 events in 297.041485ms
Burst completed: 5000 events in 268.707865ms
Burst completed: 5000 events in 397.413434ms
Burst completed: 5000 events in 290.662828ms
Burst test completed: 50000 events in 8.152801342s, errors: 0
Events/sec: 6132.86
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.646214936s
Combined ops/sec: 2028.71
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 403337 queries in 1m0.003445945s
Queries/sec: 6721.90
Avg query latency: 1.650663ms
P95 query latency: 6.533977ms
P99 query latency: 10.449883ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 319133 operations (269133 queries, 50000 writes) in 1m0.003897433s
Operations/sec: 5318.54
Avg latency: 1.45724ms
Avg query latency: 1.423521ms
Avg write latency: 1.638735ms
P95 latency: 3.643619ms
P99 latency: 5.821572ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 3.018694521s
Total Events: 50000
Events/sec: 16563.45
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 101 MB
Avg Latency: 1.32589ms
P90 Latency: 1.831543ms
P95 Latency: 2.152481ms
P99 Latency: 3.113153ms
Bottom 10% Avg Latency: 726.176µs
----------------------------------------
Test: Burst Pattern
Duration: 8.152801342s
Total Events: 50000
Events/sec: 6132.86
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 251 MB
Avg Latency: 1.340819ms
P90 Latency: 1.980055ms
P95 Latency: 2.37338ms
P99 Latency: 3.737908ms
Bottom 10% Avg Latency: 567.81µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.646214936s
Total Events: 50000
Events/sec: 2028.71
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 174 MB
Avg Latency: 387.51µs
P90 Latency: 813.774µs
P95 Latency: 904.165µs
P99 Latency: 1.114634ms
Bottom 10% Avg Latency: 1.027038ms
----------------------------------------
Test: Query Performance
Duration: 1m0.003445945s
Total Events: 403337
Events/sec: 6721.90
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 148 MB
Avg Latency: 1.650663ms
P90 Latency: 4.924325ms
P95 Latency: 6.533977ms
P99 Latency: 10.449883ms
Bottom 10% Avg Latency: 7.309323ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003897433s
Total Events: 319133
Events/sec: 5318.54
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 131 MB
Avg Latency: 1.45724ms
P90 Latency: 2.888865ms
P95 Latency: 3.643619ms
P99 Latency: 5.821572ms
Bottom 10% Avg Latency: 4.174905ms
----------------------------------------
Report saved to: /tmp/benchmark_next-orly-dgraph_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_next-orly-dgraph_8/benchmark_report.adoc
RELAY_NAME: next-orly-dgraph
RELAY_URL: ws://next-orly-dgraph:8080
TEST_TIMESTAMP: 2025-11-20T20:12:04+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_next-orly-neo4j_8
Events: 50000, Workers: 24, Duration: 1m0s
1763669529971033 migrating to version 1... /build/pkg/database/migrations.go:66
1763669529971109 migrating to version 2... /build/pkg/database/migrations.go:73
1763669529971132 migrating to version 3... /build/pkg/database/migrations.go:80
1763669529971137 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763669529971148 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763669529971161 migrating to version 4... /build/pkg/database/migrations.go:87
1763669529971166 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763669529971175 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763669529971181 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:12:09 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:12:09 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 3.41945316s
Events/sec: 14622.22
Avg latency: 1.559545ms
P90 latency: 2.247167ms
P95 latency: 2.658118ms
P99 latency: 3.995878ms
Bottom 10% Avg latency: 795.698µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 295.869274ms
Burst completed: 5000 events in 462.260099ms
Burst completed: 5000 events in 296.659792ms
Burst completed: 5000 events in 291.58686ms
Burst completed: 5000 events in 283.019359ms
Burst completed: 5000 events in 333.11738ms
Burst completed: 5000 events in 297.160854ms
Burst completed: 5000 events in 262.623572ms
Burst completed: 5000 events in 287.679452ms
Burst completed: 5000 events in 272.330641ms
Burst test completed: 50000 events in 8.087375023s, errors: 0
Events/sec: 6182.48
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.430407247s
Combined ops/sec: 2046.63
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 404255 queries in 1m0.00592055s
Queries/sec: 6736.92
Avg query latency: 1.650794ms
P95 query latency: 6.53105ms
P99 query latency: 10.385042ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 314542 operations (264542 queries, 50000 writes) in 1m0.002714905s
Operations/sec: 5242.13
Avg latency: 1.461702ms
Avg query latency: 1.440494ms
Avg write latency: 1.573909ms
P95 latency: 3.707878ms
P99 latency: 6.186047ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 3.41945316s
Total Events: 50000
Events/sec: 14622.22
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 101 MB
Avg Latency: 1.559545ms
P90 Latency: 2.247167ms
P95 Latency: 2.658118ms
P99 Latency: 3.995878ms
Bottom 10% Avg Latency: 795.698µs
----------------------------------------
Test: Burst Pattern
Duration: 8.087375023s
Total Events: 50000
Events/sec: 6182.48
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 204 MB
Avg Latency: 1.269605ms
P90 Latency: 1.879279ms
P95 Latency: 2.293256ms
P99 Latency: 3.759611ms
Bottom 10% Avg Latency: 515.108µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.430407247s
Total Events: 50000
Events/sec: 2046.63
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 178 MB
Avg Latency: 363.59µs
P90 Latency: 771.255µs
P95 Latency: 867.888µs
P99 Latency: 1.099979ms
Bottom 10% Avg Latency: 996.877µs
----------------------------------------
Test: Query Performance
Duration: 1m0.00592055s
Total Events: 404255
Events/sec: 6736.92
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 129 MB
Avg Latency: 1.650794ms
P90 Latency: 4.922944ms
P95 Latency: 6.53105ms
P99 Latency: 10.385042ms
Bottom 10% Avg Latency: 7.275184ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.002714905s
Total Events: 314542
Events/sec: 5242.13
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 132 MB
Avg Latency: 1.461702ms
P90 Latency: 2.939737ms
P95 Latency: 3.707878ms
P99 Latency: 6.186047ms
Bottom 10% Avg Latency: 4.332858ms
----------------------------------------
Report saved to: /tmp/benchmark_next-orly-neo4j_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_next-orly-neo4j_8/benchmark_report.adoc
RELAY_NAME: next-orly-neo4j
RELAY_URL: ws://next-orly-neo4j:8080
TEST_TIMESTAMP: 2025-11-20T20:15:27+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_nostr-rs-relay_8
Events: 50000, Workers: 24, Duration: 1m0s
1763670543093453 migrating to version 1... /build/pkg/database/migrations.go:66
1763670543093533 migrating to version 2... /build/pkg/database/migrations.go:73
1763670543093555 migrating to version 3... /build/pkg/database/migrations.go:80
1763670543093560 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763670543093572 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763670543093586 migrating to version 4... /build/pkg/database/migrations.go:87
1763670543093591 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763670543093614 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763670543093619 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:29:03 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:29:03 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 2.834683217s
Events/sec: 17638.66
Avg latency: 1.18563ms
P90 latency: 1.576272ms
P95 latency: 1.847889ms
P99 latency: 2.69928ms
Bottom 10% Avg latency: 646.954µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 288.243162ms
Burst completed: 5000 events in 295.639176ms
Burst completed: 5000 events in 266.183046ms
Burst completed: 5000 events in 289.772997ms
Burst completed: 5000 events in 346.857517ms
Burst completed: 5000 events in 392.30016ms
Burst completed: 5000 events in 316.952072ms
Burst completed: 5000 events in 278.495452ms
Burst completed: 5000 events in 269.495766ms
Burst completed: 5000 events in 259.647834ms
Burst test completed: 50000 events in 8.010584112s, errors: 0
Events/sec: 6241.74
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.436170149s
Combined ops/sec: 2046.15
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 420104 queries in 1m0.004812476s
Queries/sec: 7001.17
Avg query latency: 1.581786ms
P95 query latency: 6.095087ms
P99 query latency: 9.681457ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 308305 operations (258305 queries, 50000 writes) in 1m0.003332271s
Operations/sec: 5138.13
Avg latency: 1.532137ms
Avg query latency: 1.49713ms
Avg write latency: 1.712984ms
P95 latency: 3.933782ms
P99 latency: 6.685993ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 2.834683217s
Total Events: 50000
Events/sec: 17638.66
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 88 MB
Avg Latency: 1.18563ms
P90 Latency: 1.576272ms
P95 Latency: 1.847889ms
P99 Latency: 2.69928ms
Bottom 10% Avg Latency: 646.954µs
----------------------------------------
Test: Burst Pattern
Duration: 8.010584112s
Total Events: 50000
Events/sec: 6241.74
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 150 MB
Avg Latency: 1.182584ms
P90 Latency: 1.77976ms
P95 Latency: 2.120267ms
P99 Latency: 3.024349ms
Bottom 10% Avg Latency: 448.582µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.436170149s
Total Events: 50000
Events/sec: 2046.15
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 135 MB
Avg Latency: 369.8µs
P90 Latency: 773.463µs
P95 Latency: 866.51µs
P99 Latency: 1.074516ms
Bottom 10% Avg Latency: 1.00298ms
----------------------------------------
Test: Query Performance
Duration: 1m0.004812476s
Total Events: 420104
Events/sec: 7001.17
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 151 MB
Avg Latency: 1.581786ms
P90 Latency: 4.688809ms
P95 Latency: 6.095087ms
P99 Latency: 9.681457ms
Bottom 10% Avg Latency: 6.825004ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003332271s
Total Events: 308305
Events/sec: 5138.13
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 98 MB
Avg Latency: 1.532137ms
P90 Latency: 3.100785ms
P95 Latency: 3.933782ms
P99 Latency: 6.685993ms
Bottom 10% Avg Latency: 4.60825ms
----------------------------------------
Report saved to: /tmp/benchmark_nostr-rs-relay_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_nostr-rs-relay_8/benchmark_report.adoc
RELAY_NAME: nostr-rs-relay
RELAY_URL: ws://nostr-rs-relay:8080
TEST_TIMESTAMP: 2025-11-20T20:32:20+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_relayer-basic_8
Events: 50000, Workers: 24, Duration: 1m0s
1763670138131829 migrating to version 1... /build/pkg/database/migrations.go:66
1763670138131898 migrating to version 2... /build/pkg/database/migrations.go:73
1763670138131920 migrating to version 3... /build/pkg/database/migrations.go:80
1763670138131925 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763670138131932 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763670138131949 migrating to version 4... /build/pkg/database/migrations.go:87
1763670138131956 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763670138131970 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763670138131976 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:22:18 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:22:18 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 2.894695787s
Events/sec: 17272.97
Avg latency: 1.255956ms
P90 latency: 1.664187ms
P95 latency: 1.909735ms
P99 latency: 2.638381ms
Bottom 10% Avg latency: 712.498µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 283.945575ms
Burst completed: 5000 events in 292.547115ms
Burst completed: 5000 events in 265.116118ms
Burst completed: 5000 events in 293.14728ms
Burst completed: 5000 events in 279.669829ms
Burst completed: 5000 events in 336.159523ms
Burst completed: 5000 events in 425.381146ms
Burst completed: 5000 events in 307.31666ms
Burst completed: 5000 events in 282.776535ms
Burst completed: 5000 events in 280.815353ms
Burst test completed: 50000 events in 8.054248885s, errors: 0
Events/sec: 6207.90
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.441579305s
Combined ops/sec: 2045.69
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 415731 queries in 1m0.004450095s
Queries/sec: 6928.34
Avg query latency: 1.605783ms
P95 query latency: 6.196926ms
P99 query latency: 9.937346ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 322255 operations (272255 queries, 50000 writes) in 1m0.003382114s
Operations/sec: 5370.61
Avg latency: 1.423539ms
Avg query latency: 1.403109ms
Avg write latency: 1.534783ms
P95 latency: 3.538928ms
P99 latency: 5.905702ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 2.894695787s
Total Events: 50000
Events/sec: 17272.97
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 207 MB
Avg Latency: 1.255956ms
P90 Latency: 1.664187ms
P95 Latency: 1.909735ms
P99 Latency: 2.638381ms
Bottom 10% Avg Latency: 712.498µs
----------------------------------------
Test: Burst Pattern
Duration: 8.054248885s
Total Events: 50000
Events/sec: 6207.90
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 205 MB
Avg Latency: 1.21703ms
P90 Latency: 1.859279ms
P95 Latency: 2.233521ms
P99 Latency: 3.436661ms
Bottom 10% Avg Latency: 441.188µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.441579305s
Total Events: 50000
Events/sec: 2045.69
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 177 MB
Avg Latency: 375.675µs
P90 Latency: 782.189µs
P95 Latency: 871.278µs
P99 Latency: 1.106456ms
Bottom 10% Avg Latency: 1.039345ms
----------------------------------------
Test: Query Performance
Duration: 1m0.004450095s
Total Events: 415731
Events/sec: 6928.34
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 114 MB
Avg Latency: 1.605783ms
P90 Latency: 4.727348ms
P95 Latency: 6.196926ms
P99 Latency: 9.937346ms
Bottom 10% Avg Latency: 6.948373ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003382114s
Total Events: 322255
Events/sec: 5370.61
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 110 MB
Avg Latency: 1.423539ms
P90 Latency: 2.827222ms
P95 Latency: 3.538928ms
P99 Latency: 5.905702ms
Bottom 10% Avg Latency: 4.165578ms
----------------------------------------
Report saved to: /tmp/benchmark_relayer-basic_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_relayer-basic_8/benchmark_report.adoc
RELAY_NAME: relayer-basic
RELAY_URL: ws://relayer-basic:7447
TEST_TIMESTAMP: 2025-11-20T20:25:35+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,195 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_rely-sqlite_8
Events: 50000, Workers: 24, Duration: 1m0s
1763668922245115 migrating to version 1... /build/pkg/database/migrations.go:66
1763668922245170 migrating to version 2... /build/pkg/database/migrations.go:73
1763668922245193 migrating to version 3... /build/pkg/database/migrations.go:80
1763668922245198 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763668922245208 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763668922245221 migrating to version 4... /build/pkg/database/migrations.go:87
1763668922245225 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763668922245237 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763668922245243 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:02:02 INFO: Extracted embedded libsecp256k1 to /tmp/orly-libsecp256k1/libsecp256k1.so
2025/11/20 20:02:02 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:02:02 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 2.855983841s
Events/sec: 17507.10
Avg latency: 1.223582ms
P90 latency: 1.623281ms
P95 latency: 1.87223ms
P99 latency: 2.707616ms
Bottom 10% Avg latency: 698.877µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 288.827022ms
Burst completed: 5000 events in 321.067294ms
Burst completed: 5000 events in 312.273754ms
Burst completed: 5000 events in 293.093481ms
Burst completed: 5000 events in 286.553497ms
Burst completed: 5000 events in 357.201577ms
Burst completed: 5000 events in 306.752475ms
Burst completed: 5000 events in 262.736838ms
Burst completed: 5000 events in 292.763913ms
Burst completed: 5000 events in 280.351571ms
Burst test completed: 50000 events in 8.008812743s, errors: 0
Events/sec: 6243.12
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.537090509s
Combined ops/sec: 2037.73
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 405957 queries in 1m0.005924644s
Queries/sec: 6765.28
Avg query latency: 1.641153ms
P95 query latency: 6.470517ms
P99 query latency: 10.153469ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 318529 operations (268529 queries, 50000 writes) in 1m0.003008545s
Operations/sec: 5308.55
Avg latency: 1.451707ms
Avg query latency: 1.426735ms
Avg write latency: 1.585823ms
P95 latency: 3.701027ms
P99 latency: 5.870958ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 2.855983841s
Total Events: 50000
Events/sec: 17507.10
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 162 MB
Avg Latency: 1.223582ms
P90 Latency: 1.623281ms
P95 Latency: 1.87223ms
P99 Latency: 2.707616ms
Bottom 10% Avg Latency: 698.877µs
----------------------------------------
Test: Burst Pattern
Duration: 8.008812743s
Total Events: 50000
Events/sec: 6243.12
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 155 MB
Avg Latency: 1.178662ms
P90 Latency: 1.750812ms
P95 Latency: 2.046981ms
P99 Latency: 2.905169ms
Bottom 10% Avg Latency: 438.058µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.537090509s
Total Events: 50000
Events/sec: 2037.73
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 143 MB
Avg Latency: 380.772µs
P90 Latency: 793.938µs
P95 Latency: 883.507µs
P99 Latency: 1.103633ms
Bottom 10% Avg Latency: 1.040974ms
----------------------------------------
Test: Query Performance
Duration: 1m0.005924644s
Total Events: 405957
Events/sec: 6765.28
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 105 MB
Avg Latency: 1.641153ms
P90 Latency: 4.911473ms
P95 Latency: 6.470517ms
P99 Latency: 10.153469ms
Bottom 10% Avg Latency: 7.198928ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003008545s
Total Events: 318529
Events/sec: 5308.55
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 109 MB
Avg Latency: 1.451707ms
P90 Latency: 2.895473ms
P95 Latency: 3.701027ms
P99 Latency: 5.870958ms
Bottom 10% Avg Latency: 4.211348ms
----------------------------------------
Report saved to: /tmp/benchmark_rely-sqlite_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_rely-sqlite_8/benchmark_report.adoc
RELAY_NAME: rely-sqlite
RELAY_URL: ws://rely-sqlite:3334
TEST_TIMESTAMP: 2025-11-20T20:05:19+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s

View File

@@ -0,0 +1,194 @@
Starting Nostr Relay Benchmark (Badger Backend)
Data Directory: /tmp/benchmark_strfry_8
Events: 50000, Workers: 24, Duration: 1m0s
1763670340478661 migrating to version 1... /build/pkg/database/migrations.go:66
1763670340478739 migrating to version 2... /build/pkg/database/migrations.go:73
1763670340478771 migrating to version 3... /build/pkg/database/migrations.go:80
1763670340478778 cleaning up ephemeral events (kinds 20000-29999)... /build/pkg/database/migrations.go:287
1763670340478786 cleaned up 0 ephemeral events from database /build/pkg/database/migrations.go:332
1763670340478806 migrating to version 4... /build/pkg/database/migrations.go:87
1763670340478813 converting events to optimized inline storage (Reiser4 optimization)... /build/pkg/database/migrations.go:340
1763670340478835 found 0 events to convert (0 regular, 0 replaceable, 0 addressable) /build/pkg/database/migrations.go:429
1763670340478843 migration complete: converted 0 events to optimized inline storage, deleted 0 old keys /build/pkg/database/migrations.go:538
╔════════════════════════════════════════════════════════╗
║ BADGER BACKEND BENCHMARK SUITE ║
╚════════════════════════════════════════════════════════╝
=== Starting Badger benchmark ===
RunPeakThroughputTest (Badger)..
=== Peak Throughput Test ===
2025/11/20 20:25:40 WARN: Failed to load embedded library from /tmp/orly-libsecp256k1/libsecp256k1.so: Error relocating /tmp/orly-libsecp256k1/libsecp256k1.so: __fprintf_chk: symbol not found, falling back to system paths
2025/11/20 20:25:40 INFO: Successfully loaded libsecp256k1 v5.0.0 from system path: libsecp256k1.so.2
Events saved: 50000/50000 (100.0%), errors: 0
Duration: 3.175451317s
Events/sec: 15745.79
Avg latency: 1.415908ms
P90 latency: 2.004386ms
P95 latency: 2.340716ms
P99 latency: 3.348014ms
Bottom 10% Avg latency: 739.523µs
Wiping database between tests...
RunBurstPatternTest (Badger)..
=== Burst Pattern Test ===
Burst completed: 5000 events in 301.102872ms
Burst completed: 5000 events in 294.117464ms
Burst completed: 5000 events in 273.073371ms
Burst completed: 5000 events in 301.704249ms
Burst completed: 5000 events in 299.9922ms
Burst completed: 5000 events in 339.238559ms
Burst completed: 5000 events in 312.837356ms
Burst completed: 5000 events in 280.591707ms
Burst completed: 5000 events in 277.848886ms
Burst completed: 5000 events in 295.019415ms
Burst test completed: 50000 events in 7.9814445s, errors: 0
Events/sec: 6264.53
Wiping database between tests...
RunMixedReadWriteTest (Badger)..
=== Mixed Read/Write Test ===
Generating 1000 unique synthetic events (minimum 300 bytes each)...
Generated 1000 events:
Average content size: 312 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database for read tests...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Mixed test completed: 25000 writes, 25000 reads in 24.456792977s
Combined ops/sec: 2044.42
Wiping database between tests...
RunQueryTest (Badger)..
=== Query Test ===
Generating 10000 unique synthetic events (minimum 300 bytes each)...
Generated 10000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 10000 events for query tests...
Query test completed: 419503 queries in 1m0.005474925s
Queries/sec: 6991.08
Avg query latency: 1.585509ms
P95 query latency: 6.132577ms
P99 query latency: 9.715848ms
Wiping database between tests...
RunConcurrentQueryStoreTest (Badger)..
=== Concurrent Query/Store Test ===
Generating 5000 unique synthetic events (minimum 300 bytes each)...
Generated 5000 events:
Average content size: 313 bytes
All events are unique (incremental timestamps)
All events are properly signed
Pre-populating database with 5000 events for concurrent query/store test...
Generating 50000 unique synthetic events (minimum 300 bytes each)...
Generated 50000 events:
Average content size: 314 bytes
All events are unique (incremental timestamps)
All events are properly signed
Concurrent test completed: 327824 operations (277824 queries, 50000 writes) in 1m0.003814409s
Operations/sec: 5463.39
Avg latency: 1.370145ms
Avg query latency: 1.364611ms
Avg write latency: 1.400897ms
P95 latency: 3.384594ms
P99 latency: 5.290584ms
=== Badger benchmark completed ===
================================================================================
BENCHMARK REPORT
================================================================================
Test: Peak Throughput
Duration: 3.175451317s
Total Events: 50000
Events/sec: 15745.79
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 205 MB
Avg Latency: 1.415908ms
P90 Latency: 2.004386ms
P95 Latency: 2.340716ms
P99 Latency: 3.348014ms
Bottom 10% Avg Latency: 739.523µs
----------------------------------------
Test: Burst Pattern
Duration: 7.9814445s
Total Events: 50000
Events/sec: 6264.53
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 205 MB
Avg Latency: 1.153768ms
P90 Latency: 1.713633ms
P95 Latency: 2.007502ms
P99 Latency: 2.81005ms
Bottom 10% Avg Latency: 410.391µs
----------------------------------------
Test: Mixed Read/Write
Duration: 24.456792977s
Total Events: 50000
Events/sec: 2044.42
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 144 MB
Avg Latency: 365.739µs
P90 Latency: 766.479µs
P95 Latency: 855.87µs
P99 Latency: 1.053084ms
Bottom 10% Avg Latency: 1.00241ms
----------------------------------------
Test: Query Performance
Duration: 1m0.005474925s
Total Events: 419503
Events/sec: 6991.08
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 101 MB
Avg Latency: 1.585509ms
P90 Latency: 4.683097ms
P95 Latency: 6.132577ms
P99 Latency: 9.715848ms
Bottom 10% Avg Latency: 6.848119ms
----------------------------------------
Test: Concurrent Query/Store
Duration: 1m0.003814409s
Total Events: 327824
Events/sec: 5463.39
Success Rate: 100.0%
Concurrent Workers: 24
Memory Used: 143 MB
Avg Latency: 1.370145ms
P90 Latency: 2.759625ms
P95 Latency: 3.384594ms
P99 Latency: 5.290584ms
Bottom 10% Avg Latency: 3.84975ms
----------------------------------------
Report saved to: /tmp/benchmark_strfry_8/benchmark_report.txt
AsciiDoc report saved to: /tmp/benchmark_strfry_8/benchmark_report.adoc
RELAY_NAME: strfry
RELAY_URL: ws://strfry:8080
TEST_TIMESTAMP: 2025-11-20T20:28:58+00:00
BENCHMARK_CONFIG:
Events: 50000
Workers: 24
Duration: 60s