mleku/next.orly.dev
1
0
Fork 1
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Add NIP-11 relay synchronization and group management features
Some checks failed
Go / build (push) Has been cancelled
Details
Go / release (push) Has been cancelled
Details

Browse Source 
- Introduced a new `sync` package for managing NIP-11 relay information and relay group configurations.
- Implemented a cache for NIP-11 documents, allowing retrieval of relay public keys and authoritative configurations.
- Enhanced the sync manager to update peer lists based on authoritative configurations from relay group events.
- Updated event handling to incorporate policy checks during event imports, ensuring compliance with relay rules.
- Refactored various components to utilize the new `sha256-simd` package for improved performance.
- Added comprehensive tests to validate the new synchronization and group management functionalities.
- Bumped version to v0.24.1 to reflect these changes.
This commit is contained in:
mleku
2025-11-03 18:17:15 +00:00
parent e161d0e4be
commit e56bf76257
83 changed files with 3712 additions and 7417 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
cmd/aggregator/README.md
View File

@@ -287,3 +287,71 @@ This separation allows flexible output handling:
# Events piped to another program, bloom filter saved
./aggregator -npub npub1... 2>bloom_filter.txt | jq '.content'
```
## Testing
The aggregator includes comprehensive tests to ensure reliable data collection:
### Running Tests
```bash
# Run aggregator tests
go test ./cmd/aggregator
# Run all tests including aggregator
go test ./...
# Run with verbose output
go test -v ./cmd/aggregator
```
### Integration Testing
The aggregator is tested as part of the project's integration test suite:
```bash
# Run the full test suite
./scripts/test.sh
# Run benchmarks (which include aggregator performance)
./scripts/runtests.sh
```
### Example Test Usage
```bash
# Test with mock data (if available)
go test -v ./cmd/aggregator -run TestAggregator
# Test bloom filter functionality
go test -v ./cmd/aggregator -run TestBloomFilter
```
## Development
### Building from Source
```bash
# Build the aggregator binary
go build -o aggregator ./cmd/aggregator
# Build with optimizations
go build -ldflags="-s -w" -o aggregator ./cmd/aggregator
# Cross-compile for different platforms
GOOS=linux GOARCH=amd64 go build -o aggregator-linux-amd64 ./cmd/aggregator
GOOS=darwin GOARCH=arm64 go build -o aggregator-darwin-arm64 ./cmd/aggregator
```
### Code Quality
The aggregator follows Go best practices and includes:
- Comprehensive error handling
- Memory-efficient data structures
- Concurrent processing with proper synchronization
- Extensive logging for debugging
## License
This tool is part of the next.orly.dev project and follows the same licensing terms.

2
cmd/aggregator/main.go
View File

@@ -18,7 +18,7 @@ import (
"lol.mleku.dev/chk"
"lol.mleku.dev/log"
p256k1signer "p256k1.mleku.dev/signer"
"next.orly.dev/pkg/crypto/sha256"
"github.com/minio/sha256-simd"
"next.orly.dev/pkg/encoders/bech32encoding"
"next.orly.dev/pkg/encoders/event"
"next.orly.dev/pkg/encoders/filter"

101
cmd/benchmark/README.md
View File

@@ -251,6 +251,107 @@ rm -rf external/ data/ reports/
docker-compose up --build
```
## Testing
The benchmark suite includes comprehensive testing to ensure reliable performance measurements:
### Running Tests
```bash
# Run benchmark tests
go test ./cmd/benchmark
# Run all tests including benchmark
go test ./...
# Run with verbose output
go test -v ./cmd/benchmark
```
### Integration Testing
The benchmark suite is tested as part of the project's integration test suite:
```bash
# Run the full test suite
./scripts/test.sh
# Run performance benchmarks
./scripts/runtests.sh
```
### Docker-based Testing
Test the complete benchmark environment:
```bash
# Test individual relay startup
docker-compose up next-orly
# Test full benchmark suite (requires external relays)
./scripts/setup-external-relays.sh
docker-compose up --build
# Clean up test environment
docker-compose down -v
```
### Example Test Usage
```bash
# Test benchmark configuration parsing
go test -v ./cmd/benchmark -run TestConfig
# Test individual benchmark patterns
go test -v ./cmd/benchmark -run TestPeakThroughput
# Test result aggregation
go test -v ./cmd/benchmark -run TestResults
```
## Development
### Building from Source
```bash
# Build the benchmark binary
go build -o benchmark ./cmd/benchmark
# Build with optimizations
go build -ldflags="-s -w" -o benchmark ./cmd/benchmark
# Cross-compile for different platforms
GOOS=linux GOARCH=amd64 go build -o benchmark-linux-amd64 ./cmd/benchmark
```
### Adding New Benchmark Tests
1. **Extend the Benchmark struct** in `main.go`
2. **Add new test method** following existing patterns
3. **Update main() function** to call new test
4. **Update result aggregation** in `benchmark-runner.sh`
### Modifying Relay Configurations
Each relay's configuration can be customized:
- **Resource limits**: Adjust memory/CPU limits in `docker-compose.yml`
- **Database settings**: Modify configuration files in `configs/`
- **Network settings**: Update port mappings and health checks
### Debugging
```bash
# View logs for specific relay
docker-compose logs next-orly
# Run benchmark with debug output
docker-compose up --build benchmark-runner
# Check individual container health
docker-compose ps
```
## Contributing
To add support for new relay implementations:

286
cmd/relay-tester/README.md
View File

@@ -1,6 +1,38 @@
# relay-tester
A command-line tool for testing Nostr relay implementations against the NIP-01 specification and related NIPs.
A comprehensive command-line tool for testing Nostr relay implementations against the NIP-01 specification and related NIPs. This tool validates relay compliance and helps developers ensure their implementations work correctly.
## Features
- **Comprehensive Test Coverage**: Tests all major Nostr protocol features
- **NIP Compliance Validation**: Ensures relays follow Nostr Improvement Proposals
- **Flexible Testing Options**: Run all tests or focus on specific areas
- **Multiple Output Formats**: Human-readable or JSON output for automation
- **Dependency-Aware Testing**: Tests run in correct order with proper dependencies
- **Integration with Build Pipeline**: Suitable for CI/CD integration
## Installation
### From Source
```bash
# Clone the repository
git clone <repository-url>
cd next.orly.dev
# Build the relay-tester
go build -o relay-tester ./cmd/relay-tester
# Optionally install globally
sudo mv relay-tester /usr/local/bin/
```
### Using the Install Script
```bash
# Use the provided installation script
./scripts/relaytester-install.sh
```
## Usage
@@ -10,62 +42,254 @@ relay-tester -url <relay-url> [options]
## Options
- `-url` (required): Relay websocket URL (e.g., `ws://127.0.0.1:3334` or `wss://relay.example.com`)
- `-test <name>`: Run a specific test by name (default: run all tests)
- `-json`: Output results in JSON format
- `-v`: Verbose output (shows additional info for each test)
- `-list`: List all available tests and exit
| Option | Description | Default |
|--------|-------------|---------|
| `-url` | **Required.** Relay websocket URL (e.g., `ws://127.0.0.1:3334` or `wss://relay.example.com`) | - |
| `-test <name>` | Run a specific test by name | Run all tests |
| `-json` | Output results in JSON format for automation | Human-readable |
| `-v` | Verbose output (shows additional info for each test) | false |
| `-list` | List all available tests and exit | false |
| `-timeout <duration>` | Timeout for individual test operations | 30s |
## Examples
### Run all tests against a local relay:
### Basic Testing
Run all tests against a local relay:
```bash
relay-tester -url ws://127.0.0.1:3334
```
### Run all tests with verbose output:
Run all tests with verbose output:
```bash
relay-tester -url ws://127.0.0.1:3334 -v
```
### Run a specific test:
### Specific Test Execution
Run a specific test:
```bash
relay-tester -url ws://127.0.0.1:3334 -test "Publishes basic event"
```
### Output results as JSON:
```bash
relay-tester -url ws://127.0.0.1:3334 -json
```
### List all available tests:
List all available tests:
```bash
relay-tester -list
```
### Output Formats
Output results as JSON for automation:
```bash
relay-tester -url ws://127.0.0.1:3334 -json
```
### Remote Relay Testing
Test a remote relay:
```bash
relay-tester -url wss://relay.damus.io
```
Test with custom timeout:
```bash
relay-tester -url ws://127.0.0.1:3334 -timeout 60s
```
## Exit Codes
- `0`: All required tests passed
- `0`: All required tests passed - relay is compliant
- `1`: One or more required tests failed, or an error occurred
- `2`: Invalid command-line arguments
## Test Categories
The relay-tester runs tests covering:
The relay-tester runs comprehensive tests covering:
- **Basic Event Operations**: Publishing, finding by ID/author/kind/tags
- **Filtering**: Time ranges, limits, multiple filters, scrape queries
- **Replaceable Events**: Metadata and contact list replacement
- **Parameterized Replaceable Events**: Addressable events with `d` tags
- **Event Deletion**: Deletion events (NIP-09)
- **Ephemeral Events**: Event handling for ephemeral kinds
- **EOSE Handling**: End of stored events signaling
- **Event Validation**: Signature verification, ID hash verification
- **JSON Compliance**: NIP-01 JSON escape sequences
### Core Protocol (NIP-01)
## Notes
- **Basic Event Operations**:
- Publishing events
- Finding events by ID, author, kind, and tags
- Event retrieval and validation
- Tests are run in dependency order (some tests depend on others)
- Required tests must pass for the relay to be considered compliant
- Optional tests may fail without affecting overall compliance
- The tool connects to the relay using WebSocket and runs tests sequentially
- **Filtering**:
- Time range filters (`since`, `until`)
- Limit and pagination
- Multiple concurrent filters
- Scrape queries for bulk data
- **Event Types**:
- Regular events (kind 1+)
- Replaceable events (kinds 0, 3, etc.)
- Parameterized replaceable events (addressable events with `d` tags)
- Ephemeral events (kinds 20000+)
### Extended Protocol Features
- **Event Deletion (NIP-09)**: Testing deletion event handling
- **EOSE Handling**: Proper "end of stored events" signaling
- **Event Validation**: Signature verification and ID hash validation
- **JSON Compliance**: NIP-01 JSON escape sequences and formatting
### Authentication & Access Control
- **Authentication Testing**: NIP-42 AUTH command support
- **Access Control**: Testing relay-specific access rules
- **Rate Limiting**: Basic rate limit validation
## Test Results Interpretation
### Successful Tests
```
✅ Publishes basic event
✅ Finds event by ID
✅ Filters events by time range
```
### Failed Tests
```
❌ Publishes basic event: timeout waiting for OK
❌ Filters events by time range: unexpected EOSE timing
```
### JSON Output Format
```json
{
"relay_url": "ws://127.0.0.1:3334",
"timestamp": "2024-01-01T12:00:00Z",
"tests_run": 25,
"tests_passed": 23,
"tests_failed": 2,
"results": [
{
"name": "Publishes basic event",
"status": "passed",
"duration": "0.123s"
},
{
"name": "Filters events by time range",
"status": "failed",
"error": "unexpected EOSE timing",
"duration": "0.456s"
}
]
}
```
## Integration with Build Scripts
The relay-tester is integrated with the project's testing scripts:
```bash
# Test relay with default configuration
./scripts/relaytester-test.sh
# Test relay with policy enabled
ORLY_POLICY_ENABLED=true ./scripts/relaytester-test.sh
# Test relay with ACL enabled
ORLY_ACL_MODE=follows ./scripts/relaytester-test.sh
```
## Testing Strategy
### Development Testing
During development, run tests frequently:
```bash
# Quick test against local relay
go run ./cmd/relay-tester -url ws://127.0.0.1:3334
# Test specific functionality
go run ./cmd/relay-tester -url ws://127.0.0.1:3334 -test "EOSE handling"
```
### CI/CD Integration
For automated testing in CI/CD pipelines:
```bash
# JSON output for parsing
relay-tester -url $RELAY_URL -json > test_results.json
# Check exit code
if [ $? -eq 0 ]; then
echo "All tests passed!"
else
echo "Some tests failed"
cat test_results.json
exit 1
fi
```
### Performance Testing
The relay-tester can be combined with performance testing:
```bash
# Start relay
./orly &
RELAY_PID=$!
# Run compliance tests
relay-tester -url ws://127.0.0.1:3334
# Run performance tests
./scripts/runtests.sh
# Cleanup
kill $RELAY_PID
```
## Troubleshooting
### Common Issues
1. **Connection Refused**: Ensure relay is running and accessible
2. **Timeout Errors**: Increase timeout with `-timeout` flag
3. **Authentication Required**: Some relays require NIP-42 AUTH
4. **WebSocket Errors**: Check firewall and network configuration
### Debug Output
Use verbose mode for detailed information:
```bash
relay-tester -url ws://127.0.0.1:3334 -v
```
### Test Dependencies
Tests are run in dependency order. If a foundational test fails, subsequent tests may also fail. Always fix basic event publishing before debugging complex filtering.
## Development
### Running Tests
```bash
# Run relay-tester unit tests
go test ./cmd/relay-tester
# Run all tests including relay-tester
go test ./...
# Run with coverage
go test -cover ./cmd/relay-tester
```
### Adding New Tests
1. Add test case to the test suite
2. Update test dependencies if needed
3. Ensure proper error handling
4. Update documentation
## License
This tool is part of the next.orly.dev project and follows the same licensing terms.