interim-docs-update #3

mleku · 2025-12-12T08:04:02Z

mleku commented

2025-12-12 08:04:02 +00:00

No description provided.

mleku added 7 commits 2025-12-12 08:04:03 +00:00

Fix broken submodule and add import memory optimization plan cbc502a703

- Remove broken submodule reference for pkg/protocol/blossom/blossom
  and track blossom spec files as regular files instead
- Add IMPORT_MEMORY_OPTIMIZATION_PLAN.md documenting strategies to
  constrain import memory usage to ≤1.5GB through cache reduction,
  batched syncs, batch transactions, and adaptive rate limiting
- Based on test results: 2.1M events imported in 48min at 736 events/sec
  with peak memory of 6.4GB (target is 1.5GB)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add PID-controlled adaptive rate limiting plan for relay operations afa3dce1c9

- Design comprehensive rate limiting for both reads (REQ) and writes (EVENT)
- Implement PID controller with filtered derivative to avoid noise amplification
- Apply low-pass filter before derivative computation (bandpass effect)
- Add anti-windup for integral term to prevent saturation
- Support setpoint-based control (target operating point as memory fraction)
- Separate tuning parameters for read vs write operations
- Monitor database-specific metrics (Badger LSM, Neo4j transactions)
- Combine memory pressure (70%) and load level (30%) into process variable
- Include integration examples for WebSocket handlers and import loop
- Add configuration via environment variables with sensible defaults

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Implement PID-controlled adaptive rate limiting for database operations 88b0509ad8

- Add LoadMonitor interface in pkg/interfaces/loadmonitor/ for database load metrics
- Implement PIDController with filtered derivative to suppress high-frequency noise
  - Proportional (P): immediate response to current error
  - Integral (I): eliminates steady-state offset with anti-windup clamping
  - Derivative (D): rate-of-change prediction with low-pass filtering
- Create BadgerLoadMonitor tracking L0 tables, compaction score, and cache hit ratio
- Create Neo4jLoadMonitor tracking query semaphore usage and latencies
- Add AdaptiveRateLimiter combining PID controllers for reads and writes
- Configure via environment variables:
  - ORLY_RATE_LIMIT_ENABLED: enable/disable rate limiting
  - ORLY_RATE_LIMIT_TARGET_MB: target memory limit (default 1500MB)
  - ORLY_RATE_LIMIT_*_K[PID]: PID gains for reads/writes
  - ORLY_RATE_LIMIT_MAX_*_MS: maximum delays
  - ORLY_RATE_LIMIT_*_TARGET: setpoints for reads/writes
- Integrate rate limiter into Server struct and lifecycle management
- Add comprehensive unit tests for PID controller behavior

Files modified:
- app/config/config.go: Add rate limiting configuration options
- app/main.go: Initialize and start/stop rate limiter
- app/server.go: Add rateLimiter field to Server struct
- main.go: Create rate limiter with appropriate monitor
- pkg/run/run.go: Pass disabled limiter for test instances
- pkg/interfaces/loadmonitor/: New LoadMonitor interface
- pkg/ratelimit/: New PID controller and limiter implementation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Generalize PID controller as reusable library with abstract interfaces 28b41847a6

- Create pkg/interfaces/pid for generic PID controller interfaces:
  - ProcessVariable: abstract input (value + timestamp)
  - Source: provides process variable samples
  - Output: controller output with P/I/D components and clamping info
  - Controller: generic PID interface with setpoint/gains
  - Tuning: configuration struct for all PID parameters

- Create pkg/pid as standalone PID controller implementation:
  - Thread-safe with mutex protection
  - Low-pass filtered derivative to suppress high-frequency noise
  - Anti-windup on integral term
  - Configurable output clamping
  - Presets for common use cases: rate limiting, PoW difficulty,
    temperature control, motor speed

- Update pkg/ratelimit to use generic pkg/pid.Controller:
  - Limiter now uses pidif.Controller interface
  - Type assertions for monitoring/debugging state access
  - Maintains backward compatibility with existing API

The generic PID package can now be used for any dynamic adjustment
scenario beyond rate limiting, such as blockchain PoW difficulty
adjustment, temperature regulation, or motor speed control.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Fix NIP-42 AUTH compliance: always respond with OK message

Go / build-and-release (push) Has been cancelled

Details

a816737cd3

- Ensure AUTH handler always sends OK response per NIP-42 specification,
  including for parse failures (uses zero event ID with error reason)
- Add zeroEventID constant for OK responses when event ID cannot be parsed
- Document critical client guidance: clients MUST wait for OK response
  after AUTH before publishing events requiring authentication
- Update nostr skill and CLAUDE.md with NIP-42 AUTH protocol requirements
  for client developers, emphasizing OK response handling
- Add MAX_THINKING_TOKENS setting to Claude configuration

Files modified:
- app/handle-auth.go: Add OK response for AUTH parse failures
- .claude/skills/nostr/SKILL.md: Document AUTH OK response requirements
- CLAUDE.md: Add NIP-42 AUTH Protocol section for client developers
- .claude/settings.local.json: Add MAX_THINKING_TOKENS setting
- pkg/version/version: Bump to v0.34.7

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add _graph extension support to Neo4j driver

Go / build-and-release (push) Has been cancelled

Details

ba84e12ea9

- Implement TraverseFollows using Cypher path queries on FOLLOWS relationships
- Implement TraverseFollowers using reverse path traversal
- Implement FindMentions using MENTIONS relationships from p-tags
- Implement TraverseThread using REFERENCES relationships from e-tags
  with bidirectional traversal (inbound replies, outbound parents)
- Add GraphAdapter to bridge Neo4j to graph.GraphDatabase interface
- Add GraphResult type implementing graph.GraphResultI for Neo4j
- Initialize graph executor for Neo4j backend in app/main.go

The implementation uses existing Neo4j schema and relationships created
by SaveEvent() - no schema changes required. The _graph extension now
works transparently with either Badger or Neo4j backends.

Bump version to v0.35.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Interim release: documentation updates and rate limiting improvements f16ab3077f

- Add applesauce library reference documentation
- Add rate limiting test report for Badger
- Add memory monitoring for rate limiter (platform-specific implementations)
- Enhance PID-controlled adaptive rate limiting
- Update Neo4j and Badger monitors with improved load metrics
- Add docker-compose configuration
- Update README and configuration options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

mleku merged commit f16ab3077f into main

2025-12-12 08:04:26 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mleku/next.orly.dev#3