Add comprehensive documentation for CLAUDE and Nostr WebSocket skills
- Introduced CLAUDE.md to provide guidance for working with the Claude Code repository, including project overview, build commands, testing guidelines, and performance considerations. - Added INDEX.md for a structured overview of the strfry WebSocket implementation analysis, detailing document contents and usage. - Created SKILL.md for the nostr-websocket skill, covering WebSocket protocol fundamentals, connection management, and performance optimization techniques. - Included multiple reference documents for Go, C++, and Rust implementations of WebSocket patterns, enhancing the knowledge base for developers. - Updated deployment and build documentation to reflect new multi-platform capabilities and pure Go build processes. - Bumped version to reflect the addition of extensive documentation and resources for developers working with Nostr relays and WebSocket connections.
This commit is contained in:
395
CLAUDE.md
Normal file
395
CLAUDE.md
Normal file
@@ -0,0 +1,395 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
ORLY is a high-performance Nostr relay written in Go, designed for personal relays, small communities, and business deployments. It emphasizes low latency, custom cryptography optimizations, and embedded database performance.
|
||||
|
||||
**Key Technologies:**
|
||||
- **Language**: Go 1.25.3+
|
||||
- **Database**: Badger v4 (embedded key-value store)
|
||||
- **Cryptography**: Custom p8k library using purego for secp256k1 operations (no CGO)
|
||||
- **Web UI**: Svelte frontend embedded in the binary
|
||||
- **WebSocket**: gorilla/websocket for Nostr protocol
|
||||
- **Performance**: SIMD-accelerated SHA256 and hex encoding
|
||||
|
||||
## Build Commands
|
||||
|
||||
### Basic Build
|
||||
```bash
|
||||
# Build relay binary only
|
||||
go build -o orly
|
||||
|
||||
# Pure Go build (no CGO) - this is the standard approach
|
||||
CGO_ENABLED=0 go build -o orly
|
||||
```
|
||||
|
||||
### Build with Web UI
|
||||
```bash
|
||||
# Recommended: Use the provided script
|
||||
./scripts/update-embedded-web.sh
|
||||
|
||||
# Manual build
|
||||
cd app/web
|
||||
bun install
|
||||
bun run build
|
||||
cd ../../
|
||||
go build -o orly
|
||||
```
|
||||
|
||||
### Development Mode (Web UI Hot Reload)
|
||||
```bash
|
||||
# Terminal 1: Start relay with dev proxy
|
||||
export ORLY_WEB_DISABLE_EMBEDDED=true
|
||||
export ORLY_WEB_DEV_PROXY_URL=localhost:5000
|
||||
./orly &
|
||||
|
||||
# Terminal 2: Start dev server
|
||||
cd app/web && bun run dev
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Run All Tests
|
||||
```bash
|
||||
# Standard test run
|
||||
./scripts/test.sh
|
||||
|
||||
# Or manually with purego setup
|
||||
CGO_ENABLED=0 go test ./...
|
||||
|
||||
# Note: libsecp256k1.so must be available for crypto tests
|
||||
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$(pwd)/pkg/crypto/p8k"
|
||||
```
|
||||
|
||||
### Run Specific Package Tests
|
||||
```bash
|
||||
# Test database package
|
||||
cd pkg/database && go test -v ./...
|
||||
|
||||
# Test protocol package
|
||||
cd pkg/protocol && go test -v ./...
|
||||
|
||||
# Test with specific test function
|
||||
go test -v -run TestSaveEvent ./pkg/database
|
||||
```
|
||||
|
||||
### Relay Protocol Testing
|
||||
```bash
|
||||
# Test relay protocol compliance
|
||||
go run cmd/relay-tester/main.go -url ws://localhost:3334
|
||||
|
||||
# List available tests
|
||||
go run cmd/relay-tester/main.go -list
|
||||
|
||||
# Run specific test
|
||||
go run cmd/relay-tester/main.go -url ws://localhost:3334 -test "Basic Event"
|
||||
```
|
||||
|
||||
### Benchmarking
|
||||
```bash
|
||||
# Run benchmarks in specific package
|
||||
go test -bench=. -benchmem ./pkg/database
|
||||
|
||||
# Crypto benchmarks
|
||||
cd pkg/crypto/p8k && make bench
|
||||
```
|
||||
|
||||
## Running the Relay
|
||||
|
||||
### Basic Run
|
||||
```bash
|
||||
# Build and run
|
||||
go build -o orly && ./orly
|
||||
|
||||
# With environment variables
|
||||
export ORLY_LOG_LEVEL=debug
|
||||
export ORLY_PORT=3334
|
||||
./orly
|
||||
```
|
||||
|
||||
### Get Relay Identity
|
||||
```bash
|
||||
# Print relay identity secret and pubkey
|
||||
./orly identity
|
||||
```
|
||||
|
||||
### Common Configuration
|
||||
```bash
|
||||
# TLS with Let's Encrypt
|
||||
export ORLY_TLS_DOMAINS=relay.example.com
|
||||
|
||||
# Admin configuration
|
||||
export ORLY_ADMINS=npub1...
|
||||
|
||||
# Follows ACL mode
|
||||
export ORLY_ACL_MODE=follows
|
||||
|
||||
# Enable sprocket event processing
|
||||
export ORLY_SPROCKET_ENABLED=true
|
||||
|
||||
# Enable policy system
|
||||
export ORLY_POLICY_ENABLED=true
|
||||
```
|
||||
|
||||
## Code Architecture
|
||||
|
||||
### Repository Structure
|
||||
|
||||
**Root Entry Point:**
|
||||
- `main.go` - Application entry point with signal handling, profiling setup, and database initialization
|
||||
- `app/main.go` - Core relay server initialization and lifecycle management
|
||||
|
||||
**Core Packages:**
|
||||
|
||||
**`app/`** - HTTP/WebSocket server and handlers
|
||||
- `server.go` - Main Server struct and HTTP request routing
|
||||
- `handle-*.go` - Nostr protocol message handlers (EVENT, REQ, COUNT, CLOSE, AUTH, DELETE)
|
||||
- `handle-websocket.go` - WebSocket connection lifecycle and frame handling
|
||||
- `listener.go` - Network listener setup
|
||||
- `sprocket.go` - External event processing script manager
|
||||
- `publisher.go` - Event broadcast to active subscriptions
|
||||
- `payment_processor.go` - NWC integration for subscription payments
|
||||
- `blossom.go` - Blob storage service initialization
|
||||
- `web.go` - Embedded web UI serving and dev proxy
|
||||
- `config/` - Environment variable configuration using go-simpler.org/env
|
||||
|
||||
**`pkg/database/`** - Badger-based event storage
|
||||
- `database.go` - Database initialization with cache tuning
|
||||
- `save-event.go` - Event storage with index updates
|
||||
- `query-events.go` - Main query execution engine
|
||||
- `query-for-*.go` - Specialized query builders for different filter patterns
|
||||
- `indexes/` - Index key construction for efficient lookups
|
||||
- `export.go` / `import.go` - Event export/import in JSONL format
|
||||
- `subscriptions.go` - Active subscription tracking
|
||||
- `identity.go` - Relay identity key management
|
||||
- `migrations.go` - Database schema migration runner
|
||||
|
||||
**`pkg/protocol/`** - Nostr protocol implementation
|
||||
- `ws/` - WebSocket message framing and parsing
|
||||
- `auth/` - NIP-42 authentication challenge/response
|
||||
- `publish/` - Event publisher for broadcasting to subscriptions
|
||||
- `relayinfo/` - NIP-11 relay information document
|
||||
- `directory/` - Distributed directory service (NIP-XX)
|
||||
- `nwc/` - Nostr Wallet Connect client
|
||||
- `blossom/` - Blob storage protocol
|
||||
|
||||
**`pkg/encoders/`** - Optimized Nostr data encoding/decoding
|
||||
- `event/` - Event JSON marshaling/unmarshaling with buffer pooling
|
||||
- `filter/` - Filter parsing and validation
|
||||
- `bech32encoding/` - npub/nsec/note encoding
|
||||
- `hex/` - SIMD-accelerated hex encoding using templexxx/xhex
|
||||
- `timestamp/`, `kind/`, `tag/` - Specialized field encoders
|
||||
|
||||
**`pkg/crypto/`** - Cryptographic operations
|
||||
- `p8k/` - Pure Go secp256k1 using purego (no CGO) to dynamically load libsecp256k1.so
|
||||
- `secp.go` - Dynamic library loading and function binding
|
||||
- `schnorr.go` - Schnorr signature operations (NIP-01)
|
||||
- `ecdh.go` - ECDH for encrypted DMs (NIP-04, NIP-44)
|
||||
- `recovery.go` - Public key recovery from signatures
|
||||
- `libsecp256k1.so` - Pre-compiled secp256k1 library
|
||||
- `keys/` - Key derivation and conversion utilities
|
||||
- `sha256/` - SIMD-accelerated SHA256 using minio/sha256-simd
|
||||
|
||||
**`pkg/acl/`** - Access control systems
|
||||
- `acl.go` - ACL registry and interface
|
||||
- `follows.go` - Follows-based whitelist (admins + their follows can write)
|
||||
- `managed.go` - NIP-86 managed relay with role-based permissions
|
||||
- `none.go` - Open relay (no restrictions)
|
||||
|
||||
**`pkg/policy/`** - Event filtering and validation policies
|
||||
- Policy configuration loaded from `~/.config/ORLY/policy.json`
|
||||
- Per-kind size limits, age restrictions, custom scripts
|
||||
- See `docs/POLICY_USAGE_GUIDE.md` for configuration examples
|
||||
|
||||
**`pkg/sync/`** - Distributed synchronization
|
||||
- `cluster_manager.go` - Active replication between relay peers
|
||||
- `relay_group_manager.go` - Relay group configuration (NIP-XX)
|
||||
- `manager.go` - Distributed directory consensus
|
||||
|
||||
**`pkg/spider/`** - Event syncing from other relays
|
||||
- `spider.go` - Spider manager for "follows" mode
|
||||
- Fetches events from admin relays for followed pubkeys
|
||||
|
||||
**`pkg/utils/`** - Shared utilities
|
||||
- `atomic/` - Extended atomic operations
|
||||
- `interrupt/` - Signal handling and graceful shutdown
|
||||
- `apputil/` - Application-level utilities
|
||||
|
||||
**Web UI (`app/web/`):**
|
||||
- Svelte-based admin interface
|
||||
- Embedded in binary via `go:embed`
|
||||
- Features: event browser, sprocket management, user admin, settings
|
||||
|
||||
**Command-line Tools (`cmd/`):**
|
||||
- `relay-tester/` - Nostr protocol compliance testing
|
||||
- `benchmark/` - Multi-relay performance comparison
|
||||
- `stresstest/` - Load testing tool
|
||||
- `aggregator/` - Event aggregation utility
|
||||
- `convert/` - Data format conversion
|
||||
- `policytest/` - Policy validation testing
|
||||
|
||||
### Important Patterns
|
||||
|
||||
**Pure Go with Purego:**
|
||||
- All builds use `CGO_ENABLED=0`
|
||||
- The p8k crypto library uses `github.com/ebitengine/purego` to dynamically load `libsecp256k1.so` at runtime
|
||||
- This avoids CGO complexity while maintaining C library performance
|
||||
- `libsecp256k1.so` must be in `LD_LIBRARY_PATH` or same directory as binary
|
||||
|
||||
**Database Query Pattern:**
|
||||
- Filters are analyzed in `get-indexes-from-filter.go` to determine optimal query strategy
|
||||
- Different query builders (`query-for-kinds.go`, `query-for-authors.go`, etc.) handle specific filter patterns
|
||||
- All queries return event serials (uint64) for efficient joining
|
||||
- Final events fetched via `fetch-events-by-serials.go`
|
||||
|
||||
**WebSocket Message Flow:**
|
||||
1. `handle-websocket.go` accepts connection and spawns goroutine
|
||||
2. Incoming frames parsed by `pkg/protocol/ws/`
|
||||
3. Routed to handlers: `handle-event.go`, `handle-req.go`, `handle-count.go`, etc.
|
||||
4. Events stored via `database.SaveEvent()`
|
||||
5. Active subscriptions notified via `publishers.Publish()`
|
||||
|
||||
**Configuration System:**
|
||||
- Uses `go-simpler.org/env` for struct tags
|
||||
- All config in `app/config/config.go` with `ORLY_` prefix
|
||||
- Supports XDG directories via `github.com/adrg/xdg`
|
||||
- Default data directory: `~/.local/share/ORLY`
|
||||
|
||||
**Event Publishing:**
|
||||
- `pkg/protocol/publish/` manages publisher registry
|
||||
- Each WebSocket connection registers its subscriptions
|
||||
- `publishers.Publish(event)` broadcasts to matching subscribers
|
||||
- Efficient filter matching without re-querying database
|
||||
|
||||
**Embedded Assets:**
|
||||
- Web UI built to `app/web/dist/`
|
||||
- Embedded via `//go:embed` directive in `app/web.go`
|
||||
- Served at root path `/` with API at `/api/*`
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Making Changes to Web UI
|
||||
1. Edit files in `app/web/src/`
|
||||
2. For hot reload: `cd app/web && bun run dev` (with `ORLY_WEB_DISABLE_EMBEDDED=true`)
|
||||
3. For production build: `./scripts/update-embedded-web.sh`
|
||||
|
||||
### Adding New Nostr Protocol Handlers
|
||||
1. Create `app/handle-<message-type>.go`
|
||||
2. Add case in `app/handle-message.go` message router
|
||||
3. Implement handler following existing patterns
|
||||
4. Add tests in `app/<handler>_test.go`
|
||||
|
||||
### Adding Database Indexes
|
||||
1. Define index in `pkg/database/indexes/`
|
||||
2. Add migration in `pkg/database/migrations.go`
|
||||
3. Update `save-event.go` to populate index
|
||||
4. Add query builder in `pkg/database/query-for-<index>.go`
|
||||
5. Update `get-indexes-from-filter.go` to use new index
|
||||
|
||||
### Environment Variables for Development
|
||||
```bash
|
||||
# Verbose logging
|
||||
export ORLY_LOG_LEVEL=trace
|
||||
export ORLY_DB_LOG_LEVEL=debug
|
||||
|
||||
# Enable profiling
|
||||
export ORLY_PPROF=cpu
|
||||
export ORLY_PPROF_HTTP=true # Serves on :6060
|
||||
|
||||
# Health check endpoint
|
||||
export ORLY_HEALTH_PORT=8080
|
||||
```
|
||||
|
||||
### Profiling
|
||||
```bash
|
||||
# CPU profiling
|
||||
export ORLY_PPROF=cpu
|
||||
./orly
|
||||
# Profile written on shutdown
|
||||
|
||||
# HTTP pprof server
|
||||
export ORLY_PPROF_HTTP=true
|
||||
./orly
|
||||
# Visit http://localhost:6060/debug/pprof/
|
||||
|
||||
# Memory profiling
|
||||
export ORLY_PPROF=memory
|
||||
export ORLY_PPROF_PATH=/tmp/profiles
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Automated Deployment
|
||||
```bash
|
||||
# Deploy with systemd service
|
||||
./scripts/deploy.sh
|
||||
```
|
||||
|
||||
This script:
|
||||
1. Installs Go 1.25.0 if needed
|
||||
2. Builds relay with embedded web UI
|
||||
3. Installs to `~/.local/bin/orly`
|
||||
4. Creates systemd service
|
||||
5. Sets capabilities for port 443 binding
|
||||
|
||||
### systemd Service Management
|
||||
```bash
|
||||
# Start/stop/restart
|
||||
sudo systemctl start orly
|
||||
sudo systemctl stop orly
|
||||
sudo systemctl restart orly
|
||||
|
||||
# Enable on boot
|
||||
sudo systemctl enable orly
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u orly -f
|
||||
```
|
||||
|
||||
### Manual Deployment
|
||||
```bash
|
||||
# Build for production
|
||||
./scripts/update-embedded-web.sh
|
||||
|
||||
# Or build all platforms
|
||||
./scripts/build-all-platforms.sh
|
||||
```
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
- `github.com/dgraph-io/badger/v4` - Embedded database
|
||||
- `github.com/gorilla/websocket` - WebSocket server
|
||||
- `github.com/minio/sha256-simd` - SIMD SHA256
|
||||
- `github.com/templexxx/xhex` - SIMD hex encoding
|
||||
- `github.com/ebitengine/purego` - CGO-free C library loading
|
||||
- `go-simpler.org/env` - Environment variable configuration
|
||||
- `lol.mleku.dev` - Custom logging library
|
||||
|
||||
## Testing Guidelines
|
||||
|
||||
- Test files use `_test.go` suffix
|
||||
- Use `github.com/stretchr/testify` for assertions
|
||||
- Database tests require temporary database setup (see `pkg/database/testmain_test.go`)
|
||||
- WebSocket tests should use `relay-tester` package
|
||||
- Always clean up resources in tests (database, connections, goroutines)
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Database Caching**: Tune `ORLY_DB_BLOCK_CACHE_MB` and `ORLY_DB_INDEX_CACHE_MB` for workload
|
||||
- **Query Optimization**: Add indexes for common filter patterns
|
||||
- **Memory Pooling**: Use buffer pools in encoders (see `pkg/encoders/event/`)
|
||||
- **SIMD Operations**: Leverage minio/sha256-simd and templexxx/xhex
|
||||
- **Goroutine Management**: Each WebSocket connection runs in its own goroutine
|
||||
|
||||
## Release Process
|
||||
|
||||
1. Update version in `pkg/version/version` file (e.g., v1.2.3)
|
||||
2. Create and push tag:
|
||||
```bash
|
||||
git tag v1.2.3
|
||||
git push origin v1.2.3
|
||||
```
|
||||
3. GitHub Actions workflow builds binaries for multiple platforms
|
||||
4. Release created automatically with binaries and checksums
|
||||
Reference in New Issue
Block a user