Commit Graph

213 Commits

Author SHA1 Message Date
70944d45df Add extensive tests and improve policy configuration handling
Some checks failed
Go / build-and-release (push) Has been cancelled
Introduce comprehensive tests for policy validation logic, including owner and policy admin scenarios. Update `HandlePolicyConfigUpdate` to differentiate permissions for owners and policy admins, enforcing stricter field restrictions and validation flows.
2025-12-02 07:51:59 +00:00
5631c162d9 Add default security configuration and policy recipes
Introduced default security settings with stricter access control, including policies requiring owner/admin privileges by default. Added multiple pre-configured policy recipes, custom validator support, and extended documentation for security, configurations, and use cases.
2025-12-01 21:39:28 +00:00
2166ff7013 Remove subscription_stability_test.go and improve test variable naming
Some checks failed
Go / build-and-release (push) Has been cancelled
Deleted `subscription_stability_test.go` to clean up unused or redundant code. Updated naming in test files for improved readability, replacing `tag` with `tg` for consistency. Also updated the `github.com/klauspost/compress` dependency to v1.18.2.
2025-12-01 18:47:15 +00:00
869006c4c3 Add comprehensive tests for new policy fields and combinations
Some checks failed
Go / build-and-release (push) Has been cancelled
Introduce tests to validate functionality for new policy fields, including `max_expiry_duration`, `protected_required`, `identifier_regex`, and `follows_whitelist_admins`. Also, cover combinations of new and existing fields to ensure compatibility and precedence rules are correctly enforced.

bump to v0.31.2
2025-12-01 18:21:38 +00:00
48b0b6984c Fix directory spider tag loss: size limits and validation
Some checks failed
Go / build-and-release (push) Has been cancelled
- Increase WebSocket message size limit from 500KB to 10MB to prevent
  truncation of large kind 3 follow list events (8000+ follows)
- Add validation in SaveEvent to reject kind 3 events without p tags
  before storage, preventing malformed events from buggy relays
- Implement CleanupKind3WithoutPTags() to remove existing malformed
  kind 3 events at startup
- Add enhanced logging showing tag count and event ID when rejecting
  invalid kind 3 events for better observability
- Create round-trip test proving binary tag encoding preserves p tags
  correctly through JSON→binary→JSON cycle
- Root cause: 500KB limit was truncating large follow lists during
  WebSocket receive, causing tags to be lost or incomplete
- Three-layer defense: prevent at gate (size), validate (save time),
  and cleanup (startup)

Files modified:
- app/handle-websocket.go: Increase DefaultMaxMessageSize to 10MB
- pkg/database/save-event.go: Add kind 3 validation with logging
- pkg/database/cleanup-kind3.go: New cleanup function
- pkg/database/cleanup-kind3_test.go: Round-trip test
- app/main.go: Invoke cleanup at startup
2025-11-27 13:49:33 +00:00
7fedcd24d3 initial draft of hot reload policy 2025-11-27 06:31:34 +00:00
8757b41dd9 add directory spider 2025-11-27 00:02:14 +00:00
1810c8bef3 Fix binary tag value handling for e/p tags across database layer
Some checks failed
Go / build-and-release (push) Has been cancelled
- Update nostr library to v1.0.3 with improved binary tag support
- Replace tag.Value() calls with tag.ValueHex() to handle both binary and hex formats
- Add NormalizeTagValueForHash() for consistent filter tag normalization
- Update QueryPTagGraph to handle binary-encoded and hex-encoded pubkeys
- Fix tag matching in query-events.go using TagValuesMatchUsingTagMethods
- Add filter_utils.go with tag normalization helper functions
- Update delete operations in process-delete.go and neo4j/delete.go
- Fix ACL follows extraction to use ValueHex() for consistent decoding
- Add binary_tag_filter_test.go for testing tag value normalization
- Bump version to v0.30.3
2025-11-26 21:16:46 +00:00
fad39ec201 Add serve mode, fix binary tags, document CLI tools, improve Docker
Some checks failed
Go / build-and-release (push) Has been cancelled
- Add 'serve' subcommand for ephemeral RAM-based relay at /dev/shm with
  open ACL mode for testing and benchmarking
- Fix e-tag and p-tag decoding to use ValueHex()/ValueBinary() methods
  instead of Value() which returns raw bytes for binary-optimized storage
- Document all command-line tools in readme.adoc (relay-tester, benchmark,
  stresstest, blossomtest, aggregator, convert, FIND, policytest, etc.)
- Switch Docker images from Alpine to Debian for proper libsecp256k1
  Schnorr signature and ECDH support required by Nostr
- Upgrade Docker Go version from 1.21 to 1.25
- Add ramdisk mode (--ramdisk) to benchmark script for eliminating disk
  I/O bottlenecks in performance measurements
- Add docker-compose.ramdisk.yml for tmpfs-based benchmark volumes
- Add test coverage for privileged policy with binary-encoded p-tags
- Fix blossom test to expect 200 OK for anonymous uploads when auth is
  not required (RequireAuth=false with ACL mode 'none')
- Update follows ACL to handle both binary and hex p-tag formats
- Grant owner access to all users in serve mode via None ACL
- Add benchmark reports from multi-relay comparison run
- Update CLAUDE.md with binary tag handling documentation
- Bump version to v0.30.2
2025-11-26 09:52:29 +00:00
f1ddad3318 fix policy logic error caused by interface breach
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 20:46:46 +00:00
2b8f359a83 fix workflow to fetch libsecp256k1.so
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 11:04:04 +00:00
da058c37c0 blossom works fully correctly 2025-11-23 12:32:53 +00:00
1c376e6e8d migrate to new nostr library 2025-11-23 08:15:06 +00:00
00a6a78a41 fix cache to disregard subscription ids
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-20 12:30:17 +00:00
a79beee179 fixed and unified privilege checks across ACLs
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-19 13:05:21 +00:00
d4fb6cbf49 fix handleevents not prompting auth for event publish with auth-required
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-18 20:26:36 +00:00
1d4d877a10 fix auth-required not sending immediate challenge, benchmark leak 2025-11-18 18:21:11 +00:00
038d1959ed add dgraph backend to benchmark suite with safe type assertions for multi-backend support 2025-11-17 16:52:38 +00:00
511b8cae5f improve query cache with zstd level 9
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-16 20:52:18 +00:00
dfe8b5f8b2 add a filter query cache 512mb that stores already decoded recent query results
this should improve performance noticeably for typical kind 1 client queries
2025-11-16 18:29:53 +00:00
95bcf85ad7 optimizing badger cache, won a 10-15% improvement in most benchmarks 2025-11-16 15:07:36 +00:00
bf8d912063 enhance spider with rate limit handling, follow list updates, and improved reconnect logic; bump version to v0.29.0
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
also reduces CPU load for spider, and minor CORS fixes
2025-11-14 21:15:24 +00:00
24eef5b5a8 fix CORS headers and a wasm experiment
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-14 19:15:50 +00:00
29e175efb0 implement event table subtyping for small events in value log
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-14 12:15:52 +00:00
7169a2158f when in "none" ACL mode, privileged checks are not enforced
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-13 08:31:02 +00:00
cc99fcfab5 bump to v0.27.5
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 14:38:05 +00:00
d0dbd2e2dc implemented and tested NIP-43 invite based ACL 2025-11-09 10:41:58 +00:00
f0beb83ceb fix utf8 handling bug, bump to v0.26.4
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-08 10:29:24 +00:00
b4760c49b6 implement messages and operations for FIND 2025-11-08 08:54:58 +00:00
c79cd2ffee Remove deprecated files and enhance subscription stability
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Deleted obsolete files including ALL_FIXES.md, MESSAGE_QUEUE_FIX.md, PUBLISHER_FIX.md, and others to streamline the codebase.
- Implemented critical fixes for subscription stability, ensuring receiver channels are consumed and preventing drops.
- Introduced per-subscription consumer goroutines to enhance event delivery and prevent message queue overflow.
- Updated documentation to reflect changes and provide clear testing guidelines for subscription stability.
- Bumped version to v0.26.3 to signify these important updates.
2025-11-06 20:10:08 +00:00
581e0ec588 Implement comprehensive WebSocket subscription stability fixes
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Resolved critical issues causing subscriptions to drop after 30-60 seconds due to unconsumed receiver channels.
- Introduced per-subscription consumer goroutines to ensure continuous event delivery and prevent channel overflow.
- Enhanced REQ parsing to handle both wrapped and unwrapped filter arrays, eliminating EOF errors.
- Updated publisher logic to correctly send events to receiver channels, ensuring proper event delivery to subscribers.
- Added extensive documentation and testing tools to verify subscription stability and performance.
- Bumped version to v0.26.2 to reflect these significant improvements.
2025-11-06 18:21:00 +00:00
29ab350eed Implement advanced filtering capabilities in the search interface
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Added a FilterBuilder component to allow users to create complex search filters based on various criteria such as event kinds, authors, and tags.
- Introduced a FilterDisplay component to show active filters and provide an option to clear them.
- Updated the App.svelte to integrate the new filtering features, including handling filter application and clearing.
- Enhanced search functionality to utilize the new filter structure, improving the search results experience.
- Bumped version to v0.26.0 to reflect these changes.
2025-11-06 09:20:18 +00:00
88d3e3f73e Refactor authentication challenge logic in HandleCount and HandleReq
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Updated the authentication challenge conditions in both HandleCount and HandleReq functions to check for the presence of an authenticated public key.
- Introduced the schnorr package to handle public key length validation.
- Bumped version to v0.25.7 to reflect these changes.
2025-11-05 19:23:59 +00:00
f35440ed1d Refactor delete handling and improve logging in event processing
- Updated the HandleDelete function to allow delete events to be stored even if no valid targets are found, enhancing flexibility in event management.
- Improved logging to provide clearer insights when no deletions are processed, ensuring that issues are logged without blocking event acceptance.
- Refactored SHA256 validation tests and adjusted expected values to ensure accuracy in hash comparisons.
- Enhanced various test cases to improve coverage and reliability across the application.
2025-11-05 08:48:21 +00:00
9d13811f6b Add test files and enhance logging in various components
- Introduced test files for the blossom and database packages to improve test coverage and ensure functionality.
- Updated logging practices by suppressing unnecessary log outputs during tests to enhance clarity and focus on relevant information.
- Refactored error handling in the `handle-message` and `handle-req` functions to avoid logging expected context cancellation errors during shutdown.
- Bumped version to v0.25.2 to reflect these updates.
2025-11-05 08:15:02 +00:00
1d12099f1c Enhance WebSocket connection management and signer implementation
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Added a pong handler to extend the read deadline upon receiving PONG messages in WebSocket connections, improving connection stability.
- Updated the signer implementation to serialize the x-only public key to 32 bytes, ensuring compatibility with the internal format.
- Refactored tests to utilize MustNew() for signer initialization, enhancing error handling during key generation.
- Bumped version to v0.25.1 to reflect these updates.
2025-11-05 06:19:28 +00:00
e0a95ca1cd Refactor signer implementation to use p8k package
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Replaced all instances of p256k1signer with the new p8k.Signer across various modules, including event creation, policy handling, and database interactions.
- Updated related test cases and benchmarks to ensure compatibility with the new signer interface.
- Bumped version to v0.25.0 to reflect these significant changes and improvements in cryptographic operations.
2025-11-04 20:05:19 +00:00
fa71e9e334 Add Golang skill and reference materials
- Introduced a new skill for Golang, providing comprehensive guidance on writing, debugging, and best practices for Go programming.
- Added reference materials including effective Go guidelines, common patterns, and a quick reference cheat sheet to support users in Go development.
- Created a skill creator guide to assist in developing new skills with structured templates and resource management.
- Implemented scripts for skill initialization and packaging to streamline the skill creation process.
2025-11-04 12:36:36 +00:00
cefd0a98e7 Remove unused pong handler in WebSocket connection management and bump version to v0.24.6
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-04 10:44:45 +00:00
4b0dcfdf94 Add cluster replication configuration and enhance event handling
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced support for cluster replication in the ORLY system, allowing for distributed relay clusters with active replication.
- Updated the configuration to include a new option for propagating privileged events to relay peers.
- Enhanced the `ClusterManager` to manage event propagation based on the new configuration setting.
- Improved the handling of event fetching to respect the propagation settings, ensuring better privacy for privileged events.
- Updated documentation to reflect the new cluster replication features and privacy considerations.
- Bumped version to v0.24.3 to reflect these changes.
2025-11-03 19:55:14 +00:00
b1f1334e39 Add cluster replication features and membership management
- Introduced a new `ClusterManager` to handle cluster membership events and facilitate event replication across relay peers.
- Implemented HTTP endpoints for retrieving the latest serial and fetching events within a specified range.
- Enhanced event handling to process cluster membership events (Kind 39108) and update relay lists accordingly.
- Updated configuration to support cluster administrators and their management capabilities.
- Added comprehensive tests to validate the new cluster replication functionalities.
- Documented the cluster replication protocol in a new specification file.
- Bumped version to reflect these changes.
2025-11-03 19:02:20 +00:00
e56bf76257 Add NIP-11 relay synchronization and group management features
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced a new `sync` package for managing NIP-11 relay information and relay group configurations.
- Implemented a cache for NIP-11 documents, allowing retrieval of relay public keys and authoritative configurations.
- Enhanced the sync manager to update peer lists based on authoritative configurations from relay group events.
- Updated event handling to incorporate policy checks during event imports, ensuring compliance with relay rules.
- Refactored various components to utilize the new `sha256-simd` package for improved performance.
- Added comprehensive tests to validate the new synchronization and group management functionalities.
- Bumped version to v0.24.1 to reflect these changes.
2025-11-03 18:17:15 +00:00
e161d0e4be Implement distributed synchronization features
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Added a sync manager to handle distributed synchronization across relay peers, initialized in the main application run function.
- Enhanced the event handling to update the serial number for synchronization when events are processed.
- Introduced new API endpoints for synchronization, allowing peers to fetch the current serial number and events within a specified range.
- Implemented peer request validation for synchronization endpoints to ensure authorized access based on NIP-98 authentication.
- Updated configuration to support relay peers for synchronization.
- Bumped version to v0.24.0 to reflect these changes.
2025-11-03 15:54:51 +00:00
ed412dcb7e Add WebSocket workaround test and enhance connection handling
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced a new test file `workaround_test.go` to validate the behavior of a "dumb" WebSocket client that does not handle ping/pong messages correctly, ensuring the connection remains alive through server-side workarounds.
- Updated the `handle-websocket.go` file to improve message size handling and refactor ping/pong logic, allowing for direct message sending and better error management.
- Enhanced the `listener.go` file to support a more robust write channel mechanism, allowing pings to interrupt writes and improving overall connection management.
- Bumped version to v0.23.4 to reflect these changes.
2025-11-03 13:49:14 +00:00
2614b51068 Refactor crypto package to use p256k1 signer
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Replaced the p256k package with p256k1.mleku.dev/signer across the codebase, updating all instances where the previous signer was utilized.
- Removed the deprecated p256k package, including all related files and tests, to streamline the codebase and improve maintainability.
- Updated various components, including event handling, database interactions, and protocol implementations, to ensure compatibility with the new signer interface.
- Enhanced tests to validate the new signing functionality and ensure robustness across the application.
- Bumped version to v0.23.3 to reflect these changes.
2025-11-03 10:21:31 +00:00
edcdec9c7e Add Blossom blob storage server and subscription management
- Introduced the `initializeBlossomServer` function to set up the Blossom blob storage server with dynamic base URL handling and ACL configuration.
- Implemented the `blossomHandler` method to manage incoming requests to the Blossom API, ensuring proper URL handling and context management.
- Enhanced the `PaymentProcessor` to support Blossom service levels, allowing for subscription extensions based on payment metadata.
- Added methods for parsing and validating Blossom service levels, including storage quota management and subscription extension logic.
- Updated the configuration to include Blossom service level settings, facilitating dynamic service level management.
- Integrated storage quota checks in the blob upload process to prevent exceeding allocated limits.
- Refactored existing code to improve organization and maintainability, including the removal of unused blob directory configurations.
- Added tests to ensure the robustness of new functionalities and maintain existing behavior across blob operations.
2025-11-02 22:23:01 +00:00
354a2f1cda Enhance WebSocket write handling and connection management
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced a buffered write channel and a dedicated write worker goroutine to serialize write operations, preventing concurrent write panics.
- Updated the Write and WriteControl methods to send messages through the write channel, improving error handling and connection stability.
- Refactored ping and pong handlers to utilize the new write channel for sending control messages.
- Enhanced publisher logic to manage write channels for WebSocket connections, ensuring efficient message delivery and error handling.
- Bumped version to v0.23.0 to reflect these changes.
2025-11-02 17:02:28 +00:00
e56b3f0083 Refactor event handling and policy script error management
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Removed redundant log statement in HandleEvent for cleaner output.
- Enhanced policy script handling to check for script existence before execution, improving error handling and fallback logic.
- Updated error messages to provide clearer feedback when policy scripts are missing or fail to start.
- Bumped version to v0.21.2 to reflect these changes.
2025-11-01 12:55:42 +00:00
54f65d8740 Enhance relay testing and event handling
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Updated TestRelay to include a wait mechanism for relay readiness, improving test reliability.
- Refactored startTestRelay to return the assigned port, allowing dynamic port assignment.
- Added timestamp validation in HandleEvent to reject events with timestamps more than one hour in the future.
- Introduced channels for handling OK and COUNT messages in the Client struct, improving message processing.
- Updated tests to reflect changes in event timestamp handling and increased wait times for event processing.
- Bumped version to v0.20.6 to reflect these enhancements.
2025-10-30 19:12:11 +00:00
ba2d35012c Enhance WebSocket connection management and error handling
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Set initial read deadline for connections to prevent premature timeouts on idle connections.
- Improved pong and ping handlers to extend read deadlines and handle timeout errors more effectively.
- Refined error logging for connection issues, distinguishing between timeouts and connection errors to enhance debugging.
- Updated subscriber delivery logic to handle timeouts gracefully, allowing for potential recovery without immediate disconnection.
2025-10-30 18:32:03 +00:00