Some checks failed
Go / build-and-release (push) Has been cancelled
- Add 'serve' subcommand for ephemeral RAM-based relay at /dev/shm with open ACL mode for testing and benchmarking - Fix e-tag and p-tag decoding to use ValueHex()/ValueBinary() methods instead of Value() which returns raw bytes for binary-optimized storage - Document all command-line tools in readme.adoc (relay-tester, benchmark, stresstest, blossomtest, aggregator, convert, FIND, policytest, etc.) - Switch Docker images from Alpine to Debian for proper libsecp256k1 Schnorr signature and ECDH support required by Nostr - Upgrade Docker Go version from 1.21 to 1.25 - Add ramdisk mode (--ramdisk) to benchmark script for eliminating disk I/O bottlenecks in performance measurements - Add docker-compose.ramdisk.yml for tmpfs-based benchmark volumes - Add test coverage for privileged policy with binary-encoded p-tags - Fix blossom test to expect 200 OK for anonymous uploads when auth is not required (RequireAuth=false with ACL mode 'none') - Update follows ACL to handle both binary and hex p-tag formats - Grant owner access to all users in serve mode via None ACL - Add benchmark reports from multi-relay comparison run - Update CLAUDE.md with binary tag handling documentation - Bump version to v0.30.2
563 lines
17 KiB
Plaintext
563 lines
17 KiB
Plaintext
go= next.orly.dev
|
||
:toc:
|
||
:note-caption: note 👉
|
||
|
||
image:./docs/orly.png[orly.dev]
|
||
|
||
image:https://img.shields.io/badge/version-v0.24.1-blue.svg[Version v0.24.1]
|
||
image:https://img.shields.io/badge/godoc-documentation-blue.svg[Documentation,link=https://pkg.go.dev/next.orly.dev]
|
||
image:https://img.shields.io/badge/donate-geyser_crowdfunding_project_page-orange.svg[Support this project,link=https://geyser.fund/project/orly]
|
||
zap me: ⚡️mlekudev@getalby.com
|
||
follow me on link:https://jumble.social/users/npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmleku[nostr]
|
||
|
||
== about
|
||
|
||
ORLY is a nostr relay written from the ground up to be performant, low latency, and built with a number of features designed to make it well suited for
|
||
|
||
- personal relays
|
||
- small community relays
|
||
- business deployments and RaaS (Relay as a Service) with a nostr-native NWC client to allow accepting payments through NWC capable lightning nodes
|
||
- high availability clusters for reliability and/or providing a unified data set across multiple regions
|
||
|
||
== performance & cryptography
|
||
|
||
ORLY leverages high-performance libraries and custom optimizations for exceptional speed:
|
||
|
||
* **SIMD Libraries**: Uses link:https://github.com/minio/sha256-simd[minio/sha256-simd] for accelerated SHA256 hashing
|
||
* **p256k1 Cryptography**: Implements link:https://github.com/p256k1/p256k1[p256k1.mleku.dev] for fast elliptic curve operations optimized for nostr
|
||
* **Fast Message Encoders**: High-performance encoding/decoding with link:https://github.com/templexxx/xhex[templexxx/xhex] for SIMD-accelerated hex operations
|
||
|
||
The encoders achieve **24% faster JSON marshaling**, **16% faster canonical encoding**, and **54-91% reduction in memory allocations** through custom buffer pre-allocation and zero-allocation optimization techniques.
|
||
|
||
ORLY uses a fast embedded link:https://github.com/hypermodeinc/badger[badger] database with a database designed for high performance querying and event storage.
|
||
|
||
== building
|
||
|
||
ORLY is a standard Go application that can be built using the Go toolchain.
|
||
|
||
=== prerequisites
|
||
|
||
- Go 1.25.3 or later
|
||
- Git
|
||
- For web UI: link:https://bun.sh/[Bun] JavaScript runtime
|
||
|
||
=== basic build
|
||
|
||
To build the relay binary only:
|
||
|
||
[source,bash]
|
||
----
|
||
git clone <repository-url>
|
||
cd next.orly.dev
|
||
go build -o orly
|
||
----
|
||
|
||
=== building with web UI
|
||
|
||
To build with the embedded web interface:
|
||
|
||
[source,bash]
|
||
----
|
||
# Build the Svelte web application
|
||
cd app/web
|
||
bun install
|
||
bun run build
|
||
|
||
# Build the Go binary from project root
|
||
cd ../../
|
||
go build -o orly
|
||
----
|
||
|
||
The recommended way to build and embed the web UI is using the provided script:
|
||
|
||
[source,bash]
|
||
----
|
||
./scripts/update-embedded-web.sh
|
||
----
|
||
|
||
This script will:
|
||
- Build the Svelte app in `app/web` to `app/web/dist` using Bun (preferred) or fall back to npm/yarn/pnpm
|
||
- Run `go install` from the repository root so the binary picks up the new embedded assets
|
||
- Automatically detect and use the best available JavaScript package manager
|
||
|
||
For manual builds, you can also use:
|
||
|
||
[source,bash]
|
||
----
|
||
#!/bin/bash
|
||
# build.sh
|
||
echo "Building Svelte app..."
|
||
cd app/web
|
||
bun install
|
||
bun run build
|
||
|
||
echo "Building Go binary..."
|
||
cd ../../
|
||
go build -o orly
|
||
|
||
echo "Build complete!"
|
||
----
|
||
|
||
Make it executable with `chmod +x build.sh` and run with `./build.sh`.
|
||
|
||
== core features
|
||
|
||
=== web UI
|
||
|
||
ORLY includes a modern web-based user interface built with link:https://svelte.dev/[Svelte] for relay management and monitoring.
|
||
|
||
* **Secure Authentication**: Nostr key pair authentication with challenge-response
|
||
* **Event Management**: Browse, export, import, and search events
|
||
* **User Administration**: Role-based permissions (guest, user, admin, owner)
|
||
* **Sprocket Management**: Upload and monitor event processing scripts
|
||
* **Real-time Updates**: Live event streaming and system monitoring
|
||
* **Responsive Design**: Works on desktop and mobile devices
|
||
* **Dark/Light Themes**: Persistent theme preferences
|
||
|
||
The web UI is embedded in the relay binary and accessible at the relay's root path. For development with hot-reloading:
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_WEB_DISABLE_EMBEDDED=true
|
||
export ORLY_WEB_DEV_PROXY_URL=localhost:5000
|
||
./orly &
|
||
cd app/web && bun run dev
|
||
----
|
||
|
||
=== sprocket event processing
|
||
|
||
ORLY includes a powerful sprocket system for external event processing scripts. Sprocket scripts enable custom filtering, validation, and processing logic for Nostr events before storage.
|
||
|
||
* **Real-time Processing**: Scripts receive events via stdin and respond with JSONL decisions
|
||
* **Three Actions**: `accept`, `reject`, or `shadowReject` events based on custom logic
|
||
* **Automatic Recovery**: Failed scripts are automatically disabled with periodic recovery attempts
|
||
* **Web UI Management**: Upload, configure, and monitor scripts through the admin interface
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_SPROCKET_ENABLED=true
|
||
export ORLY_APP_NAME="ORLY"
|
||
# Place script at ~/.config/ORLY/sprocket.sh
|
||
----
|
||
|
||
For detailed configuration and examples, see the link:docs/sprocket/[sprocket documentation].
|
||
|
||
=== policy system
|
||
|
||
ORLY includes a comprehensive policy system for fine-grained control over event storage and retrieval. Configure custom validation rules, access controls, size limits, and age restrictions.
|
||
|
||
* **Access Control**: Allow/deny based on pubkeys, roles, or social relationships
|
||
* **Content Filtering**: Size limits, age validation, and custom rules
|
||
* **Script Integration**: Execute custom scripts for complex policy logic
|
||
* **Real-time Enforcement**: Policies applied to both read and write operations
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_POLICY_ENABLED=true
|
||
# Create policy file at ~/.config/ORLY/policy.json
|
||
----
|
||
|
||
For detailed configuration and examples, see the link:docs/POLICY_USAGE_GUIDE.md[Policy Usage Guide].
|
||
|
||
== deployment
|
||
|
||
ORLY includes an automated deployment script that handles Go installation, dependency setup, building, and systemd service configuration.
|
||
|
||
=== automated deployment
|
||
|
||
The deployment script (`scripts/deploy.sh`) provides a complete setup solution:
|
||
|
||
[source,bash]
|
||
----
|
||
# Clone the repository
|
||
git clone <repository-url>
|
||
cd next.orly.dev
|
||
|
||
# Run the deployment script
|
||
./scripts/deploy.sh
|
||
----
|
||
|
||
The script will:
|
||
|
||
1. **Install Go 1.25.3** if not present (in `~/.local/go`)
|
||
2. **Configure environment** by creating `~/.goenv` and updating `~/.bashrc`
|
||
3. **Build the relay** with embedded web UI using `update-embedded-web.sh`
|
||
4. **Set capabilities** for port 443 binding (requires sudo)
|
||
5. **Install binary** to `~/.local/bin/orly`
|
||
6. **Create systemd service** and enable it
|
||
|
||
After deployment, reload your shell environment:
|
||
|
||
[source,bash]
|
||
----
|
||
source ~/.bashrc
|
||
----
|
||
|
||
=== TLS configuration
|
||
|
||
ORLY supports automatic TLS certificate management with Let's Encrypt and custom certificates:
|
||
|
||
[source,bash]
|
||
----
|
||
# Enable TLS with Let's Encrypt for specific domains
|
||
export ORLY_TLS_DOMAINS=relay.example.com,backup.relay.example.com
|
||
|
||
# Optional: Use custom certificates (will load .pem and .key files)
|
||
export ORLY_CERTS=/path/to/cert1,/path/to/cert2
|
||
|
||
# When TLS domains are configured, ORLY will:
|
||
# - Listen on port 443 for HTTPS/WSS
|
||
# - Listen on port 80 for ACME challenges
|
||
# - Ignore ORLY_PORT setting
|
||
----
|
||
|
||
Certificate files should be named with `.pem` and `.key` extensions:
|
||
- `/path/to/cert1.pem` (certificate)
|
||
- `/path/to/cert1.key` (private key)
|
||
|
||
=== systemd service management
|
||
|
||
The deployment script creates a systemd service for easy management:
|
||
|
||
[source,bash]
|
||
----
|
||
# Start the service
|
||
sudo systemctl start orly
|
||
|
||
# Stop the service
|
||
sudo systemctl stop orly
|
||
|
||
# Restart the service
|
||
sudo systemctl restart orly
|
||
|
||
# Enable service to start on boot
|
||
sudo systemctl enable orly --now
|
||
|
||
# Disable service from starting on boot
|
||
sudo systemctl disable orly --now
|
||
|
||
# Check service status
|
||
sudo systemctl status orly
|
||
|
||
# View service logs
|
||
sudo journalctl -u orly -f
|
||
|
||
# View recent logs
|
||
sudo journalctl -u orly --since "1 hour ago"
|
||
----
|
||
|
||
=== remote deployment
|
||
|
||
You can deploy ORLY on a remote server using SSH:
|
||
|
||
[source,bash]
|
||
----
|
||
# Deploy to a VPS with SSH key authentication
|
||
ssh user@your-server.com << 'EOF'
|
||
# Clone and deploy
|
||
git clone <repository-url>
|
||
cd next.orly.dev
|
||
./scripts/deploy.sh
|
||
|
||
# Configure your relay
|
||
echo 'export ORLY_TLS_DOMAINS=relay.example.com' >> ~/.bashrc
|
||
echo 'export ORLY_ADMINS=npub1your_admin_key_here' >> ~/.bashrc
|
||
|
||
# Start the service
|
||
sudo systemctl start orly --now
|
||
EOF
|
||
|
||
# Check deployment status
|
||
ssh user@your-server.com 'sudo systemctl status orly'
|
||
----
|
||
|
||
=== configuration
|
||
|
||
After deployment, configure your relay by setting environment variables in your shell profile:
|
||
|
||
[source,bash]
|
||
----
|
||
# Add to ~/.bashrc or ~/.profile
|
||
export ORLY_TLS_DOMAINS=relay.example.com
|
||
export ORLY_ADMINS=npub1your_admin_key
|
||
export ORLY_ACL_MODE=follows
|
||
export ORLY_APP_NAME="MyRelay"
|
||
----
|
||
|
||
Then restart the service:
|
||
|
||
[source,bash]
|
||
----
|
||
source ~/.bashrc
|
||
sudo systemctl restart orly
|
||
----
|
||
|
||
=== firewall configuration
|
||
|
||
Ensure your firewall allows the necessary ports:
|
||
|
||
[source,bash]
|
||
----
|
||
# For TLS-enabled relays
|
||
sudo ufw allow 80/tcp # HTTP (ACME challenges)
|
||
sudo ufw allow 443/tcp # HTTPS/WSS
|
||
|
||
# For non-TLS relays
|
||
sudo ufw allow 3334/tcp # Default ORLY port
|
||
|
||
# Enable firewall if not already enabled
|
||
sudo ufw enable
|
||
----
|
||
|
||
=== monitoring
|
||
|
||
Monitor your relay using systemd and standard Linux tools:
|
||
|
||
[source,bash]
|
||
----
|
||
# Service status and logs
|
||
sudo systemctl status orly
|
||
sudo journalctl -u orly -f
|
||
|
||
# Resource usage
|
||
htop
|
||
sudo ss -tulpn | grep orly
|
||
|
||
# Disk usage (database grows over time)
|
||
du -sh ~/.local/share/ORLY/
|
||
|
||
# Check TLS certificates (if using Let's Encrypt)
|
||
ls -la ~/.local/share/ORLY/autocert/
|
||
----
|
||
|
||
== testing
|
||
|
||
ORLY includes comprehensive testing tools for protocol validation and performance testing.
|
||
|
||
* **Protocol Testing**: Use `relay-tester` for Nostr protocol compliance validation
|
||
* **Stress Testing**: Performance testing under various load conditions
|
||
* **Benchmark Suite**: Comparative performance testing across relay implementations
|
||
|
||
For detailed testing instructions, multi-relay testing scenarios, and advanced usage, see the link:docs/RELAY_TESTING_GUIDE.md[Relay Testing Guide].
|
||
|
||
The benchmark suite provides comprehensive performance testing and comparison across multiple relay implementations, including throughput, latency, and memory usage metrics.
|
||
|
||
== command-line tools
|
||
|
||
ORLY includes several command-line utilities in the `cmd/` directory for testing, debugging, and administration.
|
||
|
||
=== relay-tester
|
||
|
||
Nostr protocol compliance testing tool. Validates that a relay correctly implements the Nostr protocol specification.
|
||
|
||
[source,bash]
|
||
----
|
||
# Run all protocol compliance tests
|
||
go run ./cmd/relay-tester -url ws://localhost:3334
|
||
|
||
# List available tests
|
||
go run ./cmd/relay-tester -list
|
||
|
||
# Run specific test
|
||
go run ./cmd/relay-tester -url ws://localhost:3334 -test "Basic Event"
|
||
|
||
# Output results as JSON
|
||
go run ./cmd/relay-tester -url ws://localhost:3334 -json
|
||
----
|
||
|
||
=== benchmark
|
||
|
||
Comprehensive relay performance benchmarking tool. Tests event storage, queries, and subscription performance with detailed latency metrics (P90, P95, P99).
|
||
|
||
[source,bash]
|
||
----
|
||
# Run benchmarks against local database
|
||
go run ./cmd/benchmark -data-dir /tmp/bench-db -events 10000 -workers 4
|
||
|
||
# Run benchmarks against a running relay
|
||
go run ./cmd/benchmark -relay ws://localhost:3334 -events 5000
|
||
|
||
# Use different database backends
|
||
go run ./cmd/benchmark -dgraph -events 10000
|
||
go run ./cmd/benchmark -neo4j -events 10000
|
||
----
|
||
|
||
The `cmd/benchmark/` directory also includes Docker Compose configurations for comparative benchmarks across multiple relay implementations (strfry, nostr-rs-relay, khatru, etc.).
|
||
|
||
=== stresstest
|
||
|
||
Load testing tool for evaluating relay performance under sustained high-traffic conditions. Generates events with random content and tags to simulate realistic workloads.
|
||
|
||
[source,bash]
|
||
----
|
||
# Run stress test with 10 concurrent workers
|
||
go run ./cmd/stresstest -url ws://localhost:3334 -workers 10 -duration 60s
|
||
|
||
# Generate events with random p-tags (up to 100 per event)
|
||
go run ./cmd/stresstest -url ws://localhost:3334 -workers 5
|
||
----
|
||
|
||
=== blossomtest
|
||
|
||
Tests the Blossom blob storage protocol (BUD-01/BUD-02) implementation. Validates upload, download, and authentication flows.
|
||
|
||
[source,bash]
|
||
----
|
||
# Test with generated key
|
||
go run ./cmd/blossomtest -url http://localhost:3334 -size 1024
|
||
|
||
# Test with specific nsec
|
||
go run ./cmd/blossomtest -url http://localhost:3334 -nsec nsec1...
|
||
|
||
# Test anonymous uploads (no authentication)
|
||
go run ./cmd/blossomtest -url http://localhost:3334 -no-auth
|
||
----
|
||
|
||
=== aggregator
|
||
|
||
Event aggregation utility that fetches events from multiple relays using bloom filters for deduplication. Useful for syncing events across relays with memory-efficient duplicate detection.
|
||
|
||
[source,bash]
|
||
----
|
||
go run ./cmd/aggregator -relays wss://relay1.com,wss://relay2.com -output events.jsonl
|
||
----
|
||
|
||
=== convert
|
||
|
||
Key format conversion utility. Converts between hex and bech32 (npub/nsec) formats for Nostr keys.
|
||
|
||
[source,bash]
|
||
----
|
||
# Convert npub to hex
|
||
go run ./cmd/convert npub1abc...
|
||
|
||
# Convert hex to npub
|
||
go run ./cmd/convert 0123456789abcdef...
|
||
|
||
# Convert secret key (nsec or hex) - outputs both nsec and derived npub
|
||
go run ./cmd/convert --secret nsec1xyz...
|
||
----
|
||
|
||
=== FIND
|
||
|
||
Free Internet Name Daemon - CLI tool for the distributed naming system. Manages name registration, transfers, and certificate issuance.
|
||
|
||
[source,bash]
|
||
----
|
||
# Validate a name format
|
||
go run ./cmd/FIND verify-name example.nostr
|
||
|
||
# Generate a new key pair
|
||
go run ./cmd/FIND generate-key
|
||
|
||
# Create a registration proposal
|
||
go run ./cmd/FIND register myname.nostr
|
||
|
||
# Transfer a name to a new owner
|
||
go run ./cmd/FIND transfer myname.nostr npub1newowner...
|
||
----
|
||
|
||
=== policytest
|
||
|
||
Tests the policy system for event write control. Validates that policy rules correctly allow or reject events based on kind, pubkey, and other criteria.
|
||
|
||
[source,bash]
|
||
----
|
||
go run ./cmd/policytest -url ws://localhost:3334 -type event -kind 4678
|
||
go run ./cmd/policytest -url ws://localhost:3334 -type req -kind 1
|
||
go run ./cmd/policytest -url ws://localhost:3334 -type publish-and-query -count 5
|
||
----
|
||
|
||
=== policyfiltertest
|
||
|
||
Tests policy-based filtering with authorized and unauthorized pubkeys. Validates access control rules for specific users.
|
||
|
||
[source,bash]
|
||
----
|
||
go run ./cmd/policyfiltertest -url ws://localhost:3334 \
|
||
-allowed-pubkey <hex> -allowed-sec <hex> \
|
||
-unauthorized-pubkey <hex> -unauthorized-sec <hex>
|
||
----
|
||
|
||
=== subscription-test
|
||
|
||
Tests WebSocket subscription stability over extended periods. Monitors for dropped subscriptions and connection issues.
|
||
|
||
[source,bash]
|
||
----
|
||
# Run subscription stability test for 60 seconds
|
||
go run ./cmd/subscription-test -url ws://localhost:3334 -duration 60 -kind 1
|
||
|
||
# With verbose output
|
||
go run ./cmd/subscription-test -url ws://localhost:3334 -duration 120 -v
|
||
----
|
||
|
||
=== subscription-test-simple
|
||
|
||
Simplified subscription stability test that verifies subscriptions remain active without dropping over the test duration.
|
||
|
||
[source,bash]
|
||
----
|
||
go run ./cmd/subscription-test-simple -url ws://localhost:3334 -duration 120
|
||
----
|
||
|
||
== access control
|
||
|
||
=== follows ACL
|
||
|
||
The follows ACL (Access Control List) system provides flexible relay access control based on social relationships in the Nostr network.
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_ACL_MODE=follows
|
||
export ORLY_ADMINS=npub1fjqqy4a93z5zsjwsfxqhc2764kvykfdyttvldkkkdera8dr78vhsmmleku
|
||
./orly
|
||
----
|
||
|
||
The system grants write access to users followed by designated admins, with read-only access for others. Follow lists update dynamically as admins modify their relationships.
|
||
|
||
=== cluster replication
|
||
|
||
ORLY supports distributed relay clusters using active replication. When configured with peer relays, ORLY will automatically synchronize events between cluster members using efficient HTTP polling.
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_RELAY_PEERS=https://peer1.example.com,https://peer2.example.com
|
||
export ORLY_CLUSTER_ADMINS=npub1cluster_admin_key
|
||
----
|
||
|
||
**Privacy Considerations:** By default, ORLY propagates all events including privileged events (DMs, gift wraps, etc.) to cluster peers for complete synchronization. This ensures no data loss but may expose private communications to other relay operators in your cluster.
|
||
|
||
To enhance privacy, you can disable propagation of privileged events:
|
||
|
||
[source,bash]
|
||
----
|
||
export ORLY_CLUSTER_PROPAGATE_PRIVILEGED_EVENTS=false
|
||
----
|
||
|
||
**Important:** When disabled, privileged events will not be replicated to peer relays. This provides better privacy but means these events will only be available on the originating relay. Users should be aware that accessing their privileged events may require connecting directly to the relay where they were originally published.
|
||
|
||
== developer notes
|
||
|
||
=== binary-optimized tag storage
|
||
|
||
The nostr library (`git.mleku.dev/mleku/nostr/encoders/tag`) uses binary optimization for `e` and `p` tags to reduce memory usage and improve comparison performance.
|
||
|
||
When events are unmarshaled from JSON, 64-character hex values in e/p tags are converted to 33-byte binary format (32 bytes hash + null terminator).
|
||
|
||
**Important:** When working with e/p tag values in code:
|
||
|
||
* **DO NOT** use `tag.Value()` directly - it returns raw bytes which may be binary, not hex
|
||
* **ALWAYS** use `tag.ValueHex()` to get a hex string regardless of storage format
|
||
* **Use** `tag.ValueBinary()` to get raw 32-byte binary (returns nil if not binary-encoded)
|
||
|
||
[source,go]
|
||
----
|
||
// CORRECT: Use ValueHex() for hex decoding
|
||
pt, err := hex.Dec(string(pTag.ValueHex()))
|
||
|
||
// WRONG: Value() may return binary bytes, not hex
|
||
pt, err := hex.Dec(string(pTag.Value())) // Will fail for binary-encoded tags!
|
||
----
|
||
|