Compare commits

..

4 Commits

Author SHA1 Message Date
016e97925a Refactor database configuration to use centralized struct
Some checks failed
Go / build-and-release (push) Has been cancelled
Replaced individual environment variable access with a unified `DatabaseConfig` struct for all database backends. This centralizes configuration management, reduces redundant code, and ensures all options are documented in `app/config/config.go`. Backward compatibility is maintained with default values and retained constructors.
2025-12-02 13:30:50 +00:00
042b47a4d9 Make policy validation write-only and add corresponding tests
Some checks failed
Go / build-and-release (push) Has been cancelled
Updated policy validation logic to apply only to write operations, ensuring constraints like max_expiry_duration and required tags do not affect read operations. Added corresponding test cases to verify behavior for both valid and invalid inputs. This change improves clarity between write and read validation rules.

bump tag to update binary
2025-12-02 12:41:41 +00:00
952ce0285b Validate ISO-8601 duration format for max_expiry_duration
Some checks failed
Go / build-and-release (push) Has been cancelled
Added validation to reject invalid max_expiry_duration formats in policy configs, ensuring compliance with ISO-8601 standards. Updated the `New` function to fail fast on invalid inputs and included detailed error messages for better clarity. Comprehensive tests were added to verify both valid and invalid scenarios.

bump tag to build binary with update
2025-12-02 11:53:52 +00:00
45856f39b4 Update nostr to v1.0.7 with cross-platform crypto support
Some checks failed
Go / build-and-release (push) Has been cancelled
- Bump git.mleku.dev/mleku/nostr from v1.0.4 to v1.0.7
- Add p256k1.mleku.dev as indirect dependency for pure Go crypto
- Remove local replace directive for CI compatibility
- Add WASM/Mobile build plan documentation
- Bump version to v0.31.5

nostr v1.0.7 changes:
- Split crypto/p8k into platform-specific files
- Linux uses libsecp256k1 via purego (fast)
- Other platforms (darwin, windows, android) use pure Go p256k1
- Enables cross-compilation without CGO or native libraries

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 11:21:28 +00:00
16 changed files with 883 additions and 128 deletions

View File

@@ -139,9 +139,16 @@ export ORLY_SPROCKET_ENABLED=true
# Enable policy system
export ORLY_POLICY_ENABLED=true
# Database backend selection (badger or dgraph)
# Database backend selection (badger, dgraph, or neo4j)
export ORLY_DB_TYPE=badger
export ORLY_DGRAPH_URL=localhost:9080 # Only for dgraph backend
# DGraph configuration (only when ORLY_DB_TYPE=dgraph)
export ORLY_DGRAPH_URL=localhost:9080
# Neo4j configuration (only when ORLY_DB_TYPE=neo4j)
export ORLY_NEO4J_URI=bolt://localhost:7687
export ORLY_NEO4J_USER=neo4j
export ORLY_NEO4J_PASSWORD=password
# Query cache configuration (improves REQ response times)
export ORLY_QUERY_CACHE_SIZE_MB=512 # Default: 512MB
@@ -150,6 +157,7 @@ export ORLY_QUERY_CACHE_MAX_AGE=5m # Cache expiry time
# Database cache tuning (for Badger backend)
export ORLY_DB_BLOCK_CACHE_MB=512 # Block cache size
export ORLY_DB_INDEX_CACHE_MB=256 # Index cache size
export ORLY_INLINE_EVENT_THRESHOLD=1024 # Inline storage threshold (bytes)
```
## Code Architecture
@@ -299,9 +307,15 @@ export ORLY_DB_INDEX_CACHE_MB=256 # Index cache size
**Configuration System:**
- Uses `go-simpler.org/env` for struct tags
- All config in `app/config/config.go` with `ORLY_` prefix
- **IMPORTANT: ALL environment variables MUST be defined in `app/config/config.go`**
- Never use `os.Getenv()` directly in packages - always pass config via structs
- This ensures all config options appear in `./orly help` output
- Database backends receive config via `database.DatabaseConfig` struct
- Use `GetDatabaseConfigValues()` helper to extract DB config from app config
- All config fields use `ORLY_` prefix with struct tags defining defaults and usage
- Supports XDG directories via `github.com/adrg/xdg`
- Default data directory: `~/.local/share/ORLY`
- Database-specific config (Neo4j, DGraph, Badger) is passed via `DatabaseConfig` struct in `pkg/database/factory.go`
**Event Publishing:**
- `pkg/protocol/publish/` manages publisher registry

View File

@@ -1,5 +1,13 @@
// Package config provides a go-simpler.org/env configuration table and helpers
// for working with the list of key/value lists stored in .env files.
//
// IMPORTANT: This file is the SINGLE SOURCE OF TRUTH for all environment variables.
// All configuration options MUST be defined here with proper `env` struct tags.
// Never use os.Getenv() directly in other packages - pass configuration via structs.
// This ensures all options appear in `./orly help` output and are documented.
//
// For database backends, use GetDatabaseConfigValues() to extract database-specific
// settings, then construct a database.DatabaseConfig in the caller (e.g., main.go).
package config
import (
@@ -82,11 +90,19 @@ type C struct {
NIP43InviteExpiry time.Duration `env:"ORLY_NIP43_INVITE_EXPIRY" default:"24h" usage:"how long invite codes remain valid"`
// Database configuration
DBType string `env:"ORLY_DB_TYPE" default:"badger" usage:"database backend to use: badger or dgraph"`
DBType string `env:"ORLY_DB_TYPE" default:"badger" usage:"database backend to use: badger, dgraph, or neo4j"`
DgraphURL string `env:"ORLY_DGRAPH_URL" default:"localhost:9080" usage:"dgraph gRPC endpoint address (only used when ORLY_DB_TYPE=dgraph)"`
QueryCacheSizeMB int `env:"ORLY_QUERY_CACHE_SIZE_MB" default:"512" usage:"query cache size in MB (caches database query results for faster REQ responses)"`
QueryCacheMaxAge string `env:"ORLY_QUERY_CACHE_MAX_AGE" default:"5m" usage:"maximum age for cached query results (e.g., 5m, 10m, 1h)"`
// Neo4j configuration (only used when ORLY_DB_TYPE=neo4j)
Neo4jURI string `env:"ORLY_NEO4J_URI" default:"bolt://localhost:7687" usage:"Neo4j bolt URI (only used when ORLY_DB_TYPE=neo4j)"`
Neo4jUser string `env:"ORLY_NEO4J_USER" default:"neo4j" usage:"Neo4j authentication username (only used when ORLY_DB_TYPE=neo4j)"`
Neo4jPassword string `env:"ORLY_NEO4J_PASSWORD" default:"password" usage:"Neo4j authentication password (only used when ORLY_DB_TYPE=neo4j)"`
// Advanced database tuning
InlineEventThreshold int `env:"ORLY_INLINE_EVENT_THRESHOLD" default:"1024" usage:"size threshold in bytes for inline event storage in Badger (0 to disable, typical values: 384-1024)"`
// TLS configuration
TLSDomains []string `env:"ORLY_TLS_DOMAINS" usage:"comma-separated list of domains to respond to for TLS"`
Certs []string `env:"ORLY_CERTS" usage:"comma-separated list of paths to certificate root names (e.g., /path/to/cert will load /path/to/cert.pem and /path/to/cert.key)"`
@@ -369,3 +385,28 @@ func PrintHelp(cfg *C, printer io.Writer) {
PrintEnv(cfg, printer)
fmt.Fprintln(printer)
}
// GetDatabaseConfigValues returns the database configuration values as individual fields.
// This avoids circular imports with pkg/database while allowing main.go to construct
// a database.DatabaseConfig with the correct type.
func (cfg *C) GetDatabaseConfigValues() (
dataDir, logLevel string,
blockCacheMB, indexCacheMB, queryCacheSizeMB int,
queryCacheMaxAge time.Duration,
inlineEventThreshold int,
dgraphURL, neo4jURI, neo4jUser, neo4jPassword string,
) {
// Parse query cache max age from string to duration
queryCacheMaxAge = 5 * time.Minute // Default
if cfg.QueryCacheMaxAge != "" {
if duration, err := time.ParseDuration(cfg.QueryCacheMaxAge); err == nil {
queryCacheMaxAge = duration
}
}
return cfg.DataDir, cfg.DBLogLevel,
cfg.DBBlockCacheMB, cfg.DBIndexCacheMB, cfg.QueryCacheSizeMB,
queryCacheMaxAge,
cfg.InlineEventThreshold,
cfg.DgraphURL, cfg.Neo4jURI, cfg.Neo4jUser, cfg.Neo4jPassword
}

View File

@@ -177,6 +177,10 @@ LIMIT $limit
## Configuration
All configuration is centralized in `app/config/config.go` and visible via `./orly help`.
> **Important:** All environment variables must be defined in `app/config/config.go`. Do not use `os.Getenv()` directly in package code. Database backends receive configuration via the `database.DatabaseConfig` struct.
### Environment Variables
```bash

View File

@@ -0,0 +1,417 @@
# Plan: Enable js/wasm, iOS, and Android Builds
This document outlines the work required to enable ORLY and the nostr library to build successfully for WebAssembly (js/wasm), iOS (ios/arm64), and Android (android/arm64).
## Current Build Status
| Platform | Status | Notes |
|----------|--------|-------|
| linux/amd64 | ✅ Works | Uses libsecp256k1 via purego |
| darwin/arm64 | ✅ Works | Uses pure Go p256k1 |
| darwin/amd64 | ✅ Works | Uses pure Go p256k1 |
| windows/amd64 | ✅ Works | Uses pure Go p256k1 |
| android/arm64 | ✅ Works | Uses pure Go p256k1 |
| js/wasm | ❌ Fails | Missing platform stubs (planned for hackpadfs work) |
| ios/arm64 | ⚠️ Requires gomobile | See iOS section below |
---
## Issue 1: js/wasm Build Failures
### Problem
Two packages fail to compile for js/wasm due to missing platform-specific implementations:
1. **`next.orly.dev/pkg/utils/interrupt`** - Missing `Restart()` function
2. **`git.mleku.dev/mleku/nostr/ws`** - Missing `getConnectionOptions()` function
### Root Cause Analysis
#### 1.1 interrupt package
The `Restart()` function is defined with build tags:
- `restart.go``//go:build linux`
- `restart_darwin.go``//go:build darwin`
- `restart_windows.go``//go:build windows`
But `main.go` calls `Restart()` unconditionally on line 66, causing undefined symbol on js/wasm.
#### 1.2 ws package
The `getConnectionOptions()` function is defined in `connection_options.go` with:
```go
//go:build !js
```
This correctly excludes js/wasm, but no alternative implementation exists for js/wasm, so `connection.go` line 28 fails.
### Solution
#### 1.1 Fix interrupt package (ORLY)
Create a new file `restart_other.go`:
```go
//go:build !linux && !darwin && !windows
package interrupt
import (
"lol.mleku.dev/log"
"os"
)
// Restart is not supported on this platform - just exit
func Restart() {
log.W.Ln("restart not supported on this platform, exiting")
os.Exit(0)
}
```
#### 1.2 Fix ws package (nostr library)
Create a new file `connection_options_js.go`:
```go
//go:build js
package ws
import (
"crypto/tls"
"net/http"
)
// getConnectionOptions returns nil on js/wasm as we use browser WebSocket API
func getConnectionOptions(
requestHeader http.Header, tlsConfig *tls.Config,
) *websocket.Dialer {
// On js/wasm, gorilla/websocket doesn't work - need to use browser APIs
// This is a stub that allows compilation; actual WebSocket usage would
// need a js/wasm-compatible implementation
return nil
}
```
**However**, this alone won't make WebSocket work - the entire `ws` package uses `gorilla/websocket` which doesn't support js/wasm. A proper fix requires:
Option A: Use conditional compilation to swap in a js/wasm WebSocket implementation (e.g., `nhooyr.io/websocket` which supports js/wasm)
Option B: Make the `ws` package optional with build tags so js/wasm builds exclude it entirely
**Recommended**: Option B - exclude the ws client package on js/wasm since ORLY is a server, not a client.
---
## Issue 2: iOS Build Failure
### Problem
```
ios/arm64 requires external (cgo) linking, but cgo is not enabled
```
### Root Cause
iOS requires CGO for all executables due to Apple's linking requirements. This is a fundamental Go limitation - you cannot build iOS binaries with `CGO_ENABLED=0`.
### Solution
#### Option A: Accept CGO requirement for iOS
Build with CGO enabled and provide a cross-compilation toolchain:
```bash
CGO_ENABLED=1 CC=clang GOOS=ios GOARCH=arm64 go build
```
This requires:
1. Xcode with iOS SDK installed
2. Cross-compilation from macOS (or complex cross-toolchain setup)
#### Option B: Create a library instead of executable
Instead of building a standalone binary, build ORLY as a Go library that can be called from Swift/Objective-C:
```bash
CGO_ENABLED=1 GOOS=ios GOARCH=arm64 go build -buildmode=c-archive -o liborly.a
```
This creates a static library usable in iOS apps.
#### Option C: Use gomobile
Use the `gomobile` tool which handles iOS cross-compilation:
```bash
gomobile bind -target=ios ./pkg/...
```
**Recommendation**: Option A or B depending on use case. For a relay server, iOS support may not be practical anyway (iOS backgrounding restrictions, network limitations).
---
## Issue 3: Android Build Failure (RESOLVED)
### Problem
```
# github.com/ebitengine/purego
dlfcn_android.go:21:13: undefined: cgo.Dlopen
```
### Root Cause
Android uses the Linux kernel, so Go's `GOOS=android` still matches the `linux` build tag. This meant our `*_linux.go` files (which import purego) were being compiled for Android.
### Solution (Implemented)
Updated all build tags in `crypto/p8k/` to explicitly exclude Android:
**Linux files** (`*_linux.go`):
```go
//go:build linux && !android && !purego
```
**Other platform files** (`*_other.go`):
```go
//go:build !linux || android || purego
```
This ensures Android uses the pure Go `p256k1.mleku.dev` implementation instead of trying to load libsecp256k1 via purego.
### Verification
```bash
GOOS=android GOARCH=arm64 CGO_ENABLED=0 go build -o orly-android-arm64
# Successfully produces 33MB ARM64 ELF binary
```
---
## Implementation Plan
### Phase 1: js/wasm Support (Low effort)
| Task | Repository | Effort |
|------|------------|--------|
| Create `restart_other.go` stub | ORLY | 5 min |
| Create `connection_options_js.go` stub OR exclude ws package | nostr | 15 min |
| Test js/wasm build compiles | Both | 5 min |
**Note**: This enables *compilation* but not *functionality*. Running ORLY in a browser would require significant additional work (no filesystem, no listening sockets, etc.).
### Phase 2: Android Support (Medium effort)
| Task | Repository | Effort |
|------|------------|--------|
| Audit purego imports - ensure Linux-only | nostr | 30 min |
| Add build tags to any files importing purego | nostr | 15 min |
| Test android/arm64 build | Both | 5 min |
### Phase 3: iOS Support (High effort, questionable value)
| Task | Repository | Effort |
|------|------------|--------|
| Set up iOS cross-compilation environment | - | 2-4 hours |
| Modify build scripts for CGO_ENABLED=1 | ORLY | 30 min |
| Create c-archive or gomobile bindings | ORLY | 2-4 hours |
| Test on iOS simulator/device | - | 1-2 hours |
**Recommendation**: iOS support should be deprioritized unless there's a specific use case. A Nostr relay is a server, and iOS imposes severe restrictions on background network services.
---
## Quick Wins (Do First)
### 1. Create `restart_other.go` in ORLY
```go
//go:build !linux && !darwin && !windows
package interrupt
import (
"lol.mleku.dev/log"
"os"
)
func Restart() {
log.W.Ln("restart not supported on this platform, exiting")
os.Exit(0)
}
```
### 2. Exclude ws package from js/wasm in nostr library
Modify `connection.go` to have a build tag:
```go
//go:build !js
package ws
// ... rest of file
```
Create `connection_js.go`:
```go
//go:build js
package ws
// Stub package for js/wasm - WebSocket client not supported
// Use browser's native WebSocket API instead
```
### 3. Audit purego usage
Ensure all files that import `github.com/ebitengine/purego` have:
```go
//go:build linux && !purego
```
---
## Estimated Total Effort
| Platform | Compilation | Full Functionality |
|----------|-------------|-------------------|
| js/wasm | 1 hour | Not practical (server) |
| android/arm64 | 1-2 hours | Possible with NDK |
| ios/arm64 | 4-8 hours | Limited (iOS restrictions) |
---
---
## iOS with gomobile
Since iOS requires CGO and you cannot use Xcode without an Apple ID, the `gomobile` approach is the best option. This creates a framework that can be integrated into iOS apps.
### Prerequisites
1. **Install gomobile**:
```bash
go install golang.org/x/mobile/cmd/gomobile@latest
gomobile init
```
2. **Create a bindable package**:
gomobile can only bind packages that export types and functions suitable for mobile. You'll need to create a simplified API layer.
### Creating a Bindable API
Create a new package (e.g., `pkg/mobile/`) with a simplified interface:
```go
// pkg/mobile/relay.go
package mobile
import (
"context"
// ... minimal imports
)
// Relay represents an embedded Nostr relay
type Relay struct {
// internal fields
}
// NewRelay creates a new relay instance
func NewRelay(dataDir string, port int) (*Relay, error) {
// Initialize relay with mobile-friendly defaults
}
// Start begins accepting connections
func (r *Relay) Start() error {
// Start the relay server
}
// Stop gracefully shuts down the relay
func (r *Relay) Stop() error {
// Shutdown
}
// GetPublicKey returns the relay's public key
func (r *Relay) GetPublicKey() string {
// Return npub
}
```
### Building the iOS Framework
```bash
# Build iOS framework (requires macOS)
gomobile bind -target=ios -o ORLY.xcframework ./pkg/mobile
# This produces ORLY.xcframework which can be added to Xcode projects
```
### Limitations of gomobile
1. **Only certain types are bindable**:
- Basic types (int, float, string, bool, []byte)
- Structs with exported fields of bindable types
- Interfaces with methods using bindable types
- Error return values
2. **No channels or goroutines in API**:
The public API must be synchronous or use callbacks
3. **Callbacks require interfaces**:
```go
// Define callback interface
type EventHandler interface {
OnEvent(eventJSON string)
}
// Accept callback in API
func (r *Relay) SetEventHandler(h EventHandler) {
// Store and use callback
}
```
### Alternative: Building a Static Library
If you want more control, build as a C archive:
```bash
# From macOS with Xcode command line tools
CGO_ENABLED=1 GOOS=ios GOARCH=arm64 \
go build -buildmode=c-archive -o liborly.a ./pkg/mobile
# This produces:
# - liborly.a (static library)
# - liborly.h (C header file)
```
This can be linked into any iOS project using the C header.
### Recommended Next Steps for iOS
1. Create `pkg/mobile/` with a minimal, mobile-friendly API
2. Test gomobile binding on Linux first: `gomobile bind -target=android ./pkg/mobile`
3. Once Android binding works, the iOS binding will use the same API
4. Find someone with macOS to run `gomobile bind -target=ios`
---
## Appendix: File Changes Summary
### nostr Repository (`git.mleku.dev/mleku/nostr`) - COMPLETED
| File | Change |
|------|--------|
| `crypto/p8k/secp_linux.go` | Build tag: `linux && !android && !purego` |
| `crypto/p8k/schnorr_linux.go` | Build tag: `linux && !android && !purego` |
| `crypto/p8k/ecdh_linux.go` | Build tag: `linux && !android && !purego` |
| `crypto/p8k/recovery_linux.go` | Build tag: `linux && !android && !purego` |
| `crypto/p8k/utils_linux.go` | Build tag: `linux && !android && !purego` |
| `crypto/p8k/secp_other.go` | Build tag: `!linux \|\| android \|\| purego` |
| `crypto/p8k/schnorr_other.go` | Build tag: `!linux \|\| android \|\| purego` |
| `crypto/p8k/ecdh_other.go` | Build tag: `!linux \|\| android \|\| purego` |
| `crypto/p8k/recovery_other.go` | Build tag: `!linux \|\| android \|\| purego` |
| `crypto/p8k/utils_other.go` | Build tag: `!linux \|\| android \|\| purego` |
| `crypto/p8k/constants.go` | NEW - shared constants (no build tags) |
### ORLY Repository (`next.orly.dev`)
| File | Change |
|------|--------|
| `go.mod` | Added `replace` directive for local nostr library |
### Future Work (js/wasm)
| File | Action Needed |
|------|---------------|
| `pkg/utils/interrupt/restart_other.go` | CREATE - stub `Restart()` for unsupported platforms |
| `nostr/ws/connection.go` | MODIFY - add `//go:build !js` or exclude package |
| `nostr/ws/connection_js.go` | CREATE - stub for js/wasm |

3
go.mod
View File

@@ -3,7 +3,7 @@ module next.orly.dev
go 1.25.3
require (
git.mleku.dev/mleku/nostr v1.0.4
git.mleku.dev/mleku/nostr v1.0.7
github.com/adrg/xdg v0.5.3
github.com/dgraph-io/badger/v4 v4.8.0
github.com/dgraph-io/dgo/v230 v230.0.1
@@ -82,6 +82,7 @@ require (
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013 // indirect
google.golang.org/protobuf v1.36.10 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
p256k1.mleku.dev v1.0.3 // indirect
)
retract v1.0.3

6
go.sum
View File

@@ -1,6 +1,6 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
git.mleku.dev/mleku/nostr v1.0.4 h1:QKJlqUubLPeMpYpxHODSvfSlL+F6UhjBiBuze9FGRKo=
git.mleku.dev/mleku/nostr v1.0.4/go.mod h1:swI7bWLc7yU1jd7PLCCIrIcUR3Ug5O+GPvpub/w6eTY=
git.mleku.dev/mleku/nostr v1.0.7 h1:BXWsAAiGu56JXR4rIn0kaVOE+RtMmA9MPvAs8y/BjnI=
git.mleku.dev/mleku/nostr v1.0.7/go.mod h1:iYTlg2WKJXJ0kcsM6QBGOJ0UDiJidMgL/i64cHyPjZc=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
@@ -302,3 +302,5 @@ lol.mleku.dev v1.0.5/go.mod h1:JlsqP0CZDLKRyd85XGcy79+ydSRqmFkrPzYFMYxQ+zs=
lukechampine.com/frand v1.5.1 h1:fg0eRtdmGFIxhP5zQJzM1lFDbD6CUfu/f+7WgAZd5/w=
lukechampine.com/frand v1.5.1/go.mod h1:4VstaWc2plN4Mjr10chUD46RAVGWhpkZ5Nja8+Azp0Q=
nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50=
p256k1.mleku.dev v1.0.3 h1:2SBEH9XhNAotO1Ik8ejODjChTqc06Z/6ncQhrYkAdRA=
p256k1.mleku.dev v1.0.3/go.mod h1:cWkZlx6Tu7CTmIxonFbdjhdNfkY3VbjjY5TFEILiTnY=

33
main.go
View File

@@ -42,8 +42,8 @@ func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var db database.Database
if db, err = database.NewDatabase(
ctx, cancel, cfg.DBType, cfg.DataDir, cfg.DBLogLevel,
if db, err = database.NewDatabaseWithConfig(
ctx, cancel, cfg.DBType, makeDatabaseConfig(cfg),
); chk.E(err) {
os.Exit(1)
}
@@ -318,8 +318,8 @@ func main() {
ctx, cancel := context.WithCancel(context.Background())
var db database.Database
log.I.F("initializing %s database at %s", cfg.DBType, cfg.DataDir)
if db, err = database.NewDatabase(
ctx, cancel, cfg.DBType, cfg.DataDir, cfg.DBLogLevel,
if db, err = database.NewDatabaseWithConfig(
ctx, cancel, cfg.DBType, makeDatabaseConfig(cfg),
); chk.E(err) {
os.Exit(1)
}
@@ -430,3 +430,28 @@ func main() {
}
// log.I.F("exiting")
}
// makeDatabaseConfig creates a database.DatabaseConfig from the app config.
// This helper function extracts all database-specific configuration values
// and constructs the appropriate struct for the database package.
func makeDatabaseConfig(cfg *config.C) *database.DatabaseConfig {
dataDir, logLevel,
blockCacheMB, indexCacheMB, queryCacheSizeMB,
queryCacheMaxAge,
inlineEventThreshold,
dgraphURL, neo4jURI, neo4jUser, neo4jPassword := cfg.GetDatabaseConfigValues()
return &database.DatabaseConfig{
DataDir: dataDir,
LogLevel: logLevel,
BlockCacheMB: blockCacheMB,
IndexCacheMB: indexCacheMB,
QueryCacheSizeMB: queryCacheSizeMB,
QueryCacheMaxAge: queryCacheMaxAge,
InlineEventThreshold: inlineEventThreshold,
DgraphURL: dgraphURL,
Neo4jURI: neo4jURI,
Neo4jUser: neo4jUser,
Neo4jPassword: neo4jPassword,
}
}

View File

@@ -5,7 +5,6 @@ import (
"errors"
"os"
"path/filepath"
"strconv"
"time"
"github.com/dgraph-io/badger/v4"
@@ -21,10 +20,11 @@ import (
// D implements the Database interface using Badger as the storage backend
type D struct {
ctx context.Context
cancel context.CancelFunc
dataDir string
Logger *logger
ctx context.Context
cancel context.CancelFunc
dataDir string
Logger *logger
inlineEventThreshold int // Configurable threshold for inline event storage
*badger.DB
seq *badger.Sequence
pubkeySeq *badger.Sequence // Sequence for pubkey serials
@@ -35,63 +35,85 @@ type D struct {
// Ensure D implements Database interface at compile time
var _ Database = (*D)(nil)
// New creates a new Badger database instance with default configuration.
// This is provided for backward compatibility with existing callers.
// For full configuration control, use NewWithConfig instead.
func New(
ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
) (
d *D, err error,
) {
// Initialize query cache with configurable size (default 512MB)
queryCacheSize := int64(512 * 1024 * 1024) // 512 MB
if v := os.Getenv("ORLY_QUERY_CACHE_SIZE_MB"); v != "" {
if n, perr := strconv.Atoi(v); perr == nil && n > 0 {
queryCacheSize = int64(n * 1024 * 1024)
}
// Create a default config for backward compatibility
cfg := &DatabaseConfig{
DataDir: dataDir,
LogLevel: logLevel,
BlockCacheMB: 1024, // Default 1024 MB
IndexCacheMB: 512, // Default 512 MB
QueryCacheSizeMB: 512, // Default 512 MB
QueryCacheMaxAge: 5 * time.Minute, // Default 5 minutes
InlineEventThreshold: 1024, // Default 1024 bytes
}
queryCacheMaxAge := 5 * time.Minute // Default 5 minutes
if v := os.Getenv("ORLY_QUERY_CACHE_MAX_AGE"); v != "" {
if duration, perr := time.ParseDuration(v); perr == nil {
queryCacheMaxAge = duration
}
return NewWithConfig(ctx, cancel, cfg)
}
// NewWithConfig creates a new Badger database instance with full configuration.
// This is the preferred method when you have access to DatabaseConfig.
func NewWithConfig(
ctx context.Context, cancel context.CancelFunc, cfg *DatabaseConfig,
) (
d *D, err error,
) {
// Apply defaults for zero values (backward compatibility)
blockCacheMB := cfg.BlockCacheMB
if blockCacheMB == 0 {
blockCacheMB = 1024 // Default 1024 MB
}
indexCacheMB := cfg.IndexCacheMB
if indexCacheMB == 0 {
indexCacheMB = 512 // Default 512 MB
}
queryCacheSizeMB := cfg.QueryCacheSizeMB
if queryCacheSizeMB == 0 {
queryCacheSizeMB = 512 // Default 512 MB
}
queryCacheMaxAge := cfg.QueryCacheMaxAge
if queryCacheMaxAge == 0 {
queryCacheMaxAge = 5 * time.Minute // Default 5 minutes
}
inlineEventThreshold := cfg.InlineEventThreshold
if inlineEventThreshold == 0 {
inlineEventThreshold = 1024 // Default 1024 bytes
}
queryCacheSize := int64(queryCacheSizeMB * 1024 * 1024)
d = &D{
ctx: ctx,
cancel: cancel,
dataDir: dataDir,
Logger: NewLogger(lol.GetLogLevel(logLevel), dataDir),
DB: nil,
seq: nil,
ready: make(chan struct{}),
queryCache: querycache.NewEventCache(queryCacheSize, queryCacheMaxAge),
ctx: ctx,
cancel: cancel,
dataDir: cfg.DataDir,
Logger: NewLogger(lol.GetLogLevel(cfg.LogLevel), cfg.DataDir),
inlineEventThreshold: inlineEventThreshold,
DB: nil,
seq: nil,
ready: make(chan struct{}),
queryCache: querycache.NewEventCache(queryCacheSize, queryCacheMaxAge),
}
// Ensure the data directory exists
if err = os.MkdirAll(dataDir, 0755); chk.E(err) {
if err = os.MkdirAll(cfg.DataDir, 0755); chk.E(err) {
return
}
// Also ensure the directory exists using apputil.EnsureDir for any
// potential subdirectories
dummyFile := filepath.Join(dataDir, "dummy.sst")
dummyFile := filepath.Join(cfg.DataDir, "dummy.sst")
if err = apputil.EnsureDir(dummyFile); chk.E(err) {
return
}
opts := badger.DefaultOptions(d.dataDir)
// Configure caches based on environment to better match workload.
// Configure caches based on config to better match workload.
// Defaults aim for higher hit ratios under read-heavy workloads while remaining safe.
var blockCacheMB = 1024 // default 512 MB
var indexCacheMB = 512 // default 256 MB
if v := os.Getenv("ORLY_DB_BLOCK_CACHE_MB"); v != "" {
if n, perr := strconv.Atoi(v); perr == nil && n > 0 {
blockCacheMB = n
}
}
if v := os.Getenv("ORLY_DB_INDEX_CACHE_MB"); v != "" {
if n, perr := strconv.Atoi(v); perr == nil && n > 0 {
indexCacheMB = n
}
}
opts.BlockCacheSize = int64(blockCacheMB * units.Mb)
opts.IndexCacheSize = int64(indexCacheMB * units.Mb)
opts.BlockSize = 4 * units.Kb // 4 KB block size

View File

@@ -4,8 +4,33 @@ import (
"context"
"fmt"
"strings"
"time"
)
// DatabaseConfig holds all database configuration options that can be passed
// to any database backend. Each backend uses the relevant fields for its type.
// This centralizes configuration instead of having each backend read env vars directly.
type DatabaseConfig struct {
// Common settings for all backends
DataDir string
LogLevel string
// Badger-specific settings
BlockCacheMB int // ORLY_DB_BLOCK_CACHE_MB
IndexCacheMB int // ORLY_DB_INDEX_CACHE_MB
QueryCacheSizeMB int // ORLY_QUERY_CACHE_SIZE_MB
QueryCacheMaxAge time.Duration // ORLY_QUERY_CACHE_MAX_AGE
InlineEventThreshold int // ORLY_INLINE_EVENT_THRESHOLD
// DGraph-specific settings
DgraphURL string // ORLY_DGRAPH_URL
// Neo4j-specific settings
Neo4jURI string // ORLY_NEO4J_URI
Neo4jUser string // ORLY_NEO4J_USER
Neo4jPassword string // ORLY_NEO4J_PASSWORD
}
// NewDatabase creates a database instance based on the specified type.
// Supported types: "badger", "dgraph", "neo4j"
func NewDatabase(
@@ -14,19 +39,39 @@ func NewDatabase(
dbType string,
dataDir string,
logLevel string,
) (Database, error) {
// Create a default config for backward compatibility with existing callers
cfg := &DatabaseConfig{
DataDir: dataDir,
LogLevel: logLevel,
}
return NewDatabaseWithConfig(ctx, cancel, dbType, cfg)
}
// NewDatabaseWithConfig creates a database instance with full configuration.
// This is the preferred method when you have access to the app config.
func NewDatabaseWithConfig(
ctx context.Context,
cancel context.CancelFunc,
dbType string,
cfg *DatabaseConfig,
) (Database, error) {
switch strings.ToLower(dbType) {
case "badger", "":
// Use the existing badger implementation
return New(ctx, cancel, dataDir, logLevel)
return NewWithConfig(ctx, cancel, cfg)
case "dgraph":
// Use the new dgraph implementation
// Import dynamically to avoid import cycles
return newDgraphDatabase(ctx, cancel, dataDir, logLevel)
// Use the dgraph implementation
if newDgraphDatabase == nil {
return nil, fmt.Errorf("dgraph database backend not available (import _ \"next.orly.dev/pkg/dgraph\")")
}
return newDgraphDatabase(ctx, cancel, cfg)
case "neo4j":
// Use the new neo4j implementation
// Import dynamically to avoid import cycles
return newNeo4jDatabase(ctx, cancel, dataDir, logLevel)
// Use the neo4j implementation
if newNeo4jDatabase == nil {
return nil, fmt.Errorf("neo4j database backend not available (import _ \"next.orly.dev/pkg/neo4j\")")
}
return newNeo4jDatabase(ctx, cancel, cfg)
default:
return nil, fmt.Errorf("unsupported database type: %s (supported: badger, dgraph, neo4j)", dbType)
}
@@ -34,20 +79,20 @@ func NewDatabase(
// newDgraphDatabase creates a dgraph database instance
// This is defined here to avoid import cycles
var newDgraphDatabase func(context.Context, context.CancelFunc, string, string) (Database, error)
var newDgraphDatabase func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error)
// RegisterDgraphFactory registers the dgraph database factory
// This is called from the dgraph package's init() function
func RegisterDgraphFactory(factory func(context.Context, context.CancelFunc, string, string) (Database, error)) {
func RegisterDgraphFactory(factory func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error)) {
newDgraphDatabase = factory
}
// newNeo4jDatabase creates a neo4j database instance
// This is defined here to avoid import cycles
var newNeo4jDatabase func(context.Context, context.CancelFunc, string, string) (Database, error)
var newNeo4jDatabase func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error)
// RegisterNeo4jFactory registers the neo4j database factory
// This is called from the neo4j package's init() function
func RegisterNeo4jFactory(factory func(context.Context, context.CancelFunc, string, string) (Database, error)) {
func RegisterNeo4jFactory(factory func(context.Context, context.CancelFunc, *DatabaseConfig) (Database, error)) {
newNeo4jDatabase = factory
}

View File

@@ -5,8 +5,6 @@ import (
"context"
"errors"
"fmt"
"os"
"strconv"
"strings"
"github.com/dgraph-io/badger/v4"
@@ -270,14 +268,9 @@ func (d *D) SaveEvent(c context.Context, ev *event.E) (
eventData := eventDataBuf.Bytes()
// Determine storage strategy (Reiser4 optimizations)
// Get threshold from environment, default to 0 (disabled)
// When enabled, typical values: 384 (conservative), 512 (recommended), 1024 (aggressive)
smallEventThreshold := 1024
if v := os.Getenv("ORLY_INLINE_EVENT_THRESHOLD"); v != "" {
if n, perr := strconv.Atoi(v); perr == nil && n >= 0 {
smallEventThreshold = n
}
}
// Use the threshold from database configuration
// Typical values: 384 (conservative), 512 (recommended), 1024 (aggressive)
smallEventThreshold := d.inlineEventThreshold
isSmallEvent := smallEventThreshold > 0 && len(eventData) <= smallEventThreshold
isReplaceableEvent := kind.IsReplaceable(ev.Kind)
isAddressableEvent := kind.IsParameterizedReplaceable(ev.Kind)

View File

@@ -48,30 +48,21 @@ func init() {
database.RegisterDgraphFactory(func(
ctx context.Context,
cancel context.CancelFunc,
dataDir string,
logLevel string,
cfg *database.DatabaseConfig,
) (database.Database, error) {
return New(ctx, cancel, dataDir, logLevel)
return NewWithConfig(ctx, cancel, cfg)
})
}
// Config holds configuration options for the Dgraph database
type Config struct {
DataDir string
LogLevel string
DgraphURL string // Dgraph gRPC endpoint (e.g., "localhost:9080")
EnableGraphQL bool
EnableIntrospection bool
}
// New creates a new Dgraph-based database instance
func New(
ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
// NewWithConfig creates a new Dgraph-based database instance with full configuration.
// Configuration is passed from the centralized app config via DatabaseConfig.
func NewWithConfig(
ctx context.Context, cancel context.CancelFunc, cfg *database.DatabaseConfig,
) (
d *D, err error,
) {
// Get dgraph URL from environment, default to localhost
dgraphURL := os.Getenv("ORLY_DGRAPH_URL")
// Apply defaults for empty values
dgraphURL := cfg.DgraphURL
if dgraphURL == "" {
dgraphURL = "localhost:9080"
}
@@ -79,8 +70,8 @@ func New(
d = &D{
ctx: ctx,
cancel: cancel,
dataDir: dataDir,
Logger: NewLogger(lol.GetLogLevel(logLevel), dataDir),
dataDir: cfg.DataDir,
Logger: NewLogger(lol.GetLogLevel(cfg.LogLevel), cfg.DataDir),
dgraphURL: dgraphURL,
enableGraphQL: false,
enableIntrospection: false,
@@ -88,12 +79,12 @@ func New(
}
// Ensure the data directory exists
if err = os.MkdirAll(dataDir, 0755); chk.E(err) {
if err = os.MkdirAll(cfg.DataDir, 0755); chk.E(err) {
return
}
// Ensure directory structure
dummyFile := filepath.Join(dataDir, "dummy.sst")
dummyFile := filepath.Join(cfg.DataDir, "dummy.sst")
if err = apputil.EnsureDir(dummyFile); chk.E(err) {
return
}
@@ -128,6 +119,21 @@ func New(
return
}
// New creates a new Dgraph-based database instance with default configuration.
// This is provided for backward compatibility with existing callers (tests, etc.).
// For full configuration control, use NewWithConfig instead.
func New(
ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
) (
d *D, err error,
) {
cfg := &database.DatabaseConfig{
DataDir: dataDir,
LogLevel: logLevel,
}
return NewWithConfig(ctx, cancel, cfg)
}
// initDgraphClient establishes connection to dgraph server
func (d *D) initDgraphClient() error {
d.Logger.Infof("connecting to dgraph at %s", d.dgraphURL)

View File

@@ -15,6 +15,8 @@ docker run -d --name neo4j \
### 2. Configure Environment
All Neo4j configuration is defined in `app/config/config.go` and visible via `./orly help`:
```bash
export ORLY_DB_TYPE=neo4j
export ORLY_NEO4J_URI=bolt://localhost:7687
@@ -22,6 +24,8 @@ export ORLY_NEO4J_USER=neo4j
export ORLY_NEO4J_PASSWORD=password
```
> **Note:** Configuration is centralized in `app/config/config.go`. Do not use `os.Getenv()` directly in package code - all environment variables should be passed via the `database.DatabaseConfig` struct.
### 3. Run ORLY
```bash

View File

@@ -70,38 +70,29 @@ func init() {
database.RegisterNeo4jFactory(func(
ctx context.Context,
cancel context.CancelFunc,
dataDir string,
logLevel string,
cfg *database.DatabaseConfig,
) (database.Database, error) {
return New(ctx, cancel, dataDir, logLevel)
return NewWithConfig(ctx, cancel, cfg)
})
}
// Config holds configuration options for the Neo4j database
type Config struct {
DataDir string
LogLevel string
Neo4jURI string // Neo4j bolt URI (e.g., "bolt://localhost:7687")
Neo4jUser string // Authentication username
Neo4jPassword string // Authentication password
}
// New creates a new Neo4j-based database instance
func New(
ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
// NewWithConfig creates a new Neo4j-based database instance with full configuration.
// Configuration is passed from the centralized app config via DatabaseConfig.
func NewWithConfig(
ctx context.Context, cancel context.CancelFunc, cfg *database.DatabaseConfig,
) (
n *N, err error,
) {
// Get Neo4j connection details from environment
neo4jURI := os.Getenv("ORLY_NEO4J_URI")
// Apply defaults for empty values
neo4jURI := cfg.Neo4jURI
if neo4jURI == "" {
neo4jURI = "bolt://localhost:7687"
}
neo4jUser := os.Getenv("ORLY_NEO4J_USER")
neo4jUser := cfg.Neo4jUser
if neo4jUser == "" {
neo4jUser = "neo4j"
}
neo4jPassword := os.Getenv("ORLY_NEO4J_PASSWORD")
neo4jPassword := cfg.Neo4jPassword
if neo4jPassword == "" {
neo4jPassword = "password"
}
@@ -109,8 +100,8 @@ func New(
n = &N{
ctx: ctx,
cancel: cancel,
dataDir: dataDir,
Logger: NewLogger(lol.GetLogLevel(logLevel), dataDir),
dataDir: cfg.DataDir,
Logger: NewLogger(lol.GetLogLevel(cfg.LogLevel), cfg.DataDir),
neo4jURI: neo4jURI,
neo4jUser: neo4jUser,
neo4jPassword: neo4jPassword,
@@ -118,12 +109,12 @@ func New(
}
// Ensure the data directory exists
if err = os.MkdirAll(dataDir, 0755); chk.E(err) {
if err = os.MkdirAll(cfg.DataDir, 0755); chk.E(err) {
return
}
// Ensure directory structure
dummyFile := filepath.Join(dataDir, "dummy.sst")
dummyFile := filepath.Join(cfg.DataDir, "dummy.sst")
if err = apputil.EnsureDir(dummyFile); chk.E(err) {
return
}
@@ -158,6 +149,21 @@ func New(
return
}
// New creates a new Neo4j-based database instance with default configuration.
// This is provided for backward compatibility with existing callers (tests, etc.).
// For full configuration control, use NewWithConfig instead.
func New(
ctx context.Context, cancel context.CancelFunc, dataDir, logLevel string,
) (
n *N, err error,
) {
cfg := &database.DatabaseConfig{
DataDir: dataDir,
LogLevel: logLevel,
}
return NewWithConfig(ctx, cancel, cfg)
}
// initNeo4jClient establishes connection to Neo4j server
func (n *N) initNeo4jClient() error {
n.Logger.Infof("connecting to neo4j at %s", n.neo4jURI)

View File

@@ -113,11 +113,11 @@ func TestMaxExpiryDuration(t *testing.T) {
expectAllow: true,
},
{
name: "valid expiry at exact limit",
name: "expiry at exact limit rejected",
maxExpiryDuration: "PT1H",
eventExpiry: 3600, // exactly 1 hour
eventExpiry: 3600, // exactly 1 hour - >= means this is rejected
hasExpiryTag: true,
expectAllow: true,
expectAllow: false,
},
{
name: "expiry exceeds limit",
@@ -235,6 +235,79 @@ func TestMaxExpiryDurationPrecedence(t *testing.T) {
}
}
// Test that max_expiry_duration only applies to writes, not reads
func TestMaxExpiryDurationWriteOnly(t *testing.T) {
signer, pubkey := generateTestKeypair(t)
// Policy with strict max_expiry_duration
policyJSON := []byte(`{
"default_policy": "allow",
"rules": {
"4": {
"description": "DM events with expiry",
"max_expiry_duration": "PT10M",
"privileged": true
}
}
}`)
policy, err := New(policyJSON)
if err != nil {
t.Fatalf("Failed to create policy: %v", err)
}
// Create event WITHOUT an expiry tag - this would fail write validation
// but should still be readable
ev := createTestEventForNewFields(t, signer, "test DM", 4)
if err := ev.Sign(signer); chk.E(err) {
t.Fatalf("Failed to sign: %v", err)
}
// Write should fail (no expiry tag when max_expiry_duration is set)
allowed, err := policy.CheckPolicy("write", ev, pubkey, "127.0.0.1")
if err != nil {
t.Fatalf("CheckPolicy write error: %v", err)
}
if allowed {
t.Error("Write should be denied for event without expiry tag when max_expiry_duration is set")
}
// Read should succeed (validation constraints don't apply to reads)
allowed, err = policy.CheckPolicy("read", ev, pubkey, "127.0.0.1")
if err != nil {
t.Fatalf("CheckPolicy read error: %v", err)
}
if !allowed {
t.Error("Read should be allowed - max_expiry_duration is write-only validation")
}
// Also test with an event that has expiry exceeding the limit
ev2 := createTestEventForNewFields(t, signer, "test DM 2", 4)
expiryTs := ev2.CreatedAt + 7200 // 2 hours - exceeds 10 minute limit
addTagString(ev2, "expiration", int64ToString(expiryTs))
if err := ev2.Sign(signer); chk.E(err) {
t.Fatalf("Failed to sign: %v", err)
}
// Write should fail (expiry exceeds limit)
allowed, err = policy.CheckPolicy("write", ev2, pubkey, "127.0.0.1")
if err != nil {
t.Fatalf("CheckPolicy write error: %v", err)
}
if allowed {
t.Error("Write should be denied for event with expiry exceeding max_expiry_duration")
}
// Read should still succeed
allowed, err = policy.CheckPolicy("read", ev2, pubkey, "127.0.0.1")
if err != nil {
t.Fatalf("CheckPolicy read error: %v", err)
}
if !allowed {
t.Error("Read should be allowed - max_expiry_duration is write-only validation")
}
}
// =============================================================================
// ProtectedRequired Tests
// =============================================================================
@@ -1071,6 +1144,94 @@ func TestNewFieldsInGlobalRule(t *testing.T) {
}
}
// =============================================================================
// New() Validation Tests - Ensures invalid configs fail at load time
// =============================================================================
// TestNewRejectsInvalidMaxExpiryDuration verifies that New() fails fast when
// given an invalid max_expiry_duration format like "T10M" instead of "PT10M".
// This prevents silent failures where constraints are ignored.
func TestNewRejectsInvalidMaxExpiryDuration(t *testing.T) {
tests := []struct {
name string
json string
expectError bool
errorMatch string
}{
{
name: "valid PT10M format accepted",
json: `{
"rules": {
"4": {"max_expiry_duration": "PT10M"}
}
}`,
expectError: false,
},
{
name: "invalid T10M format (missing P prefix) rejected",
json: `{
"rules": {
"4": {"max_expiry_duration": "T10M"}
}
}`,
expectError: true,
errorMatch: "max_expiry_duration",
},
{
name: "invalid 10M format (missing PT prefix) rejected",
json: `{
"rules": {
"4": {"max_expiry_duration": "10M"}
}
}`,
expectError: true,
errorMatch: "max_expiry_duration",
},
{
name: "valid P7D format accepted",
json: `{
"rules": {
"1": {"max_expiry_duration": "P7D"}
}
}`,
expectError: false,
},
{
name: "invalid 7D format (missing P prefix) rejected",
json: `{
"rules": {
"1": {"max_expiry_duration": "7D"}
}
}`,
expectError: true,
errorMatch: "max_expiry_duration",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
policy, err := New([]byte(tt.json))
if tt.expectError {
if err == nil {
t.Errorf("New() should have rejected invalid config, but returned policy: %+v", policy)
return
}
if tt.errorMatch != "" && !contains(err.Error(), tt.errorMatch) {
t.Errorf("Error %q should contain %q", err.Error(), tt.errorMatch)
}
} else {
if err != nil {
t.Errorf("New() unexpected error for valid config: %v", err)
}
if policy == nil {
t.Error("New() returned nil policy for valid config")
}
}
})
}
}
// =============================================================================
// ValidateJSON Tests for New Fields
// =============================================================================

View File

@@ -468,11 +468,19 @@ func (p *P) UnmarshalJSON(data []byte) error {
// New creates a new policy from JSON configuration.
// If policyJSON is empty, returns a policy with default settings.
// The default_policy field defaults to "allow" if not specified.
// Returns an error if the policy JSON contains invalid values (e.g., invalid
// ISO-8601 duration format for max_expiry_duration, invalid regex patterns, etc.).
func New(policyJSON []byte) (p *P, err error) {
p = &P{
DefaultPolicy: "allow", // Set default value
}
if len(policyJSON) > 0 {
// Validate JSON before loading to fail fast on invalid configurations.
// This prevents silent failures where invalid values (like "T10M" instead
// of "PT10M" for max_expiry_duration) are ignored and constraints don't apply.
if err = p.ValidateJSON(policyJSON); err != nil {
return nil, fmt.Errorf("policy validation failed: %v", err)
}
if err = json.Unmarshal(policyJSON, p); chk.E(err) {
return nil, fmt.Errorf("failed to unmarshal policy JSON: %v", err)
}
@@ -1272,7 +1280,8 @@ func (p *P) checkRulePolicy(
}
// Check required tags
if len(rule.MustHaveTags) > 0 {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && len(rule.MustHaveTags) > 0 {
for _, requiredTag := range rule.MustHaveTags {
if ev.Tags.GetFirst([]byte(requiredTag)) == nil {
return false, nil
@@ -1281,7 +1290,8 @@ func (p *P) checkRulePolicy(
}
// Check expiry time (uses maxExpirySeconds which is parsed from MaxExpiryDuration or MaxExpiry)
if rule.maxExpirySeconds != nil && *rule.maxExpirySeconds > 0 {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && rule.maxExpirySeconds != nil && *rule.maxExpirySeconds > 0 {
expiryTag := ev.Tags.GetFirst([]byte("expiration"))
if expiryTag == nil {
return false, nil // Must have expiry if max_expiry is set
@@ -1294,7 +1304,7 @@ func (p *P) checkRulePolicy(
return false, nil // Invalid expiry format
}
maxAllowedExpiry := ev.CreatedAt + *rule.maxExpirySeconds
if expiryTs > maxAllowedExpiry {
if expiryTs >= maxAllowedExpiry {
log.D.F("expiration %d exceeds max allowed %d (created_at %d + max_expiry %d)",
expiryTs, maxAllowedExpiry, ev.CreatedAt, *rule.maxExpirySeconds)
return false, nil // Expiry too far in the future
@@ -1302,7 +1312,8 @@ func (p *P) checkRulePolicy(
}
// Check ProtectedRequired (NIP-70: events must have "-" tag)
if rule.ProtectedRequired {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && rule.ProtectedRequired {
protectedTag := ev.Tags.GetFirst([]byte("-"))
if protectedTag == nil {
log.D.F("protected_required: event missing '-' tag (NIP-70)")
@@ -1311,7 +1322,8 @@ func (p *P) checkRulePolicy(
}
// Check IdentifierRegex (validates "d" tag values)
if rule.identifierRegexCache != nil {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && rule.identifierRegexCache != nil {
dTags := ev.Tags.GetAll([]byte("d"))
if len(dTags) == 0 {
log.D.F("identifier_regex: event missing 'd' tag")
@@ -1328,7 +1340,8 @@ func (p *P) checkRulePolicy(
}
// Check MaxAgeOfEvent (maximum age of event in seconds)
if rule.MaxAgeOfEvent != nil && *rule.MaxAgeOfEvent > 0 {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && rule.MaxAgeOfEvent != nil && *rule.MaxAgeOfEvent > 0 {
currentTime := time.Now().Unix()
maxAllowedTime := currentTime - *rule.MaxAgeOfEvent
if ev.CreatedAt < maxAllowedTime {
@@ -1337,7 +1350,8 @@ func (p *P) checkRulePolicy(
}
// Check MaxAgeEventInFuture (maximum time event can be in the future in seconds)
if rule.MaxAgeEventInFuture != nil && *rule.MaxAgeEventInFuture > 0 {
// Only apply for write access - we validate what goes in, not what comes out
if access == "write" && rule.MaxAgeEventInFuture != nil && *rule.MaxAgeEventInFuture > 0 {
currentTime := time.Now().Unix()
maxFutureTime := currentTime + *rule.MaxAgeEventInFuture
if ev.CreatedAt > maxFutureTime {
@@ -1784,7 +1798,7 @@ func (p *P) ValidateJSON(policyJSON []byte) error {
// Validate MaxExpiryDuration format
if rule.MaxExpiryDuration != "" {
if _, err := parseDuration(rule.MaxExpiryDuration); err != nil {
return fmt.Errorf("invalid max_expiry_duration %q in kind %d: %v", rule.MaxExpiryDuration, kind, err)
return fmt.Errorf("invalid max_expiry_duration %q in kind %d: %v (format must be ISO-8601 duration, e.g. \"PT10M\" for 10 minutes, \"P7D\" for 7 days, \"P1DT12H\" for 1 day 12 hours)", rule.MaxExpiryDuration, kind, err)
}
}
// Validate FollowsWhitelistAdmins pubkeys
@@ -1815,7 +1829,7 @@ func (p *P) ValidateJSON(policyJSON []byte) error {
// Validate global rule MaxExpiryDuration format
if tempPolicy.Global.MaxExpiryDuration != "" {
if _, err := parseDuration(tempPolicy.Global.MaxExpiryDuration); err != nil {
return fmt.Errorf("invalid max_expiry_duration %q in global rule: %v", tempPolicy.Global.MaxExpiryDuration, err)
return fmt.Errorf("invalid max_expiry_duration %q in global rule: %v (format must be ISO-8601 duration, e.g. \"PT10M\" for 10 minutes, \"P7D\" for 7 days, \"P1DT12H\" for 1 day 12 hours)", tempPolicy.Global.MaxExpiryDuration, err)
}
}

View File

@@ -1 +1 @@
v0.31.4
v0.31.8