Commit Graph

162 Commits

Author SHA1 Message Date
48b0b6984c Fix directory spider tag loss: size limits and validation
Some checks failed
Go / build-and-release (push) Has been cancelled
- Increase WebSocket message size limit from 500KB to 10MB to prevent
  truncation of large kind 3 follow list events (8000+ follows)
- Add validation in SaveEvent to reject kind 3 events without p tags
  before storage, preventing malformed events from buggy relays
- Implement CleanupKind3WithoutPTags() to remove existing malformed
  kind 3 events at startup
- Add enhanced logging showing tag count and event ID when rejecting
  invalid kind 3 events for better observability
- Create round-trip test proving binary tag encoding preserves p tags
  correctly through JSON→binary→JSON cycle
- Root cause: 500KB limit was truncating large follow lists during
  WebSocket receive, causing tags to be lost or incomplete
- Three-layer defense: prevent at gate (size), validate (save time),
  and cleanup (startup)

Files modified:
- app/handle-websocket.go: Increase DefaultMaxMessageSize to 10MB
- pkg/database/save-event.go: Add kind 3 validation with logging
- pkg/database/cleanup-kind3.go: New cleanup function
- pkg/database/cleanup-kind3_test.go: Round-trip test
- app/main.go: Invoke cleanup at startup
2025-11-27 13:49:33 +00:00
5fbe131755 bump to v0.31.0 directory spider
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-27 00:03:23 +00:00
1810c8bef3 Fix binary tag value handling for e/p tags across database layer
Some checks failed
Go / build-and-release (push) Has been cancelled
- Update nostr library to v1.0.3 with improved binary tag support
- Replace tag.Value() calls with tag.ValueHex() to handle both binary and hex formats
- Add NormalizeTagValueForHash() for consistent filter tag normalization
- Update QueryPTagGraph to handle binary-encoded and hex-encoded pubkeys
- Fix tag matching in query-events.go using TagValuesMatchUsingTagMethods
- Add filter_utils.go with tag normalization helper functions
- Update delete operations in process-delete.go and neo4j/delete.go
- Fix ACL follows extraction to use ValueHex() for consistent decoding
- Add binary_tag_filter_test.go for testing tag value normalization
- Bump version to v0.30.3
2025-11-26 21:16:46 +00:00
fad39ec201 Add serve mode, fix binary tags, document CLI tools, improve Docker
Some checks failed
Go / build-and-release (push) Has been cancelled
- Add 'serve' subcommand for ephemeral RAM-based relay at /dev/shm with
  open ACL mode for testing and benchmarking
- Fix e-tag and p-tag decoding to use ValueHex()/ValueBinary() methods
  instead of Value() which returns raw bytes for binary-optimized storage
- Document all command-line tools in readme.adoc (relay-tester, benchmark,
  stresstest, blossomtest, aggregator, convert, FIND, policytest, etc.)
- Switch Docker images from Alpine to Debian for proper libsecp256k1
  Schnorr signature and ECDH support required by Nostr
- Upgrade Docker Go version from 1.21 to 1.25
- Add ramdisk mode (--ramdisk) to benchmark script for eliminating disk
  I/O bottlenecks in performance measurements
- Add docker-compose.ramdisk.yml for tmpfs-based benchmark volumes
- Add test coverage for privileged policy with binary-encoded p-tags
- Fix blossom test to expect 200 OK for anonymous uploads when auth is
  not required (RequireAuth=false with ACL mode 'none')
- Update follows ACL to handle both binary and hex p-tag formats
- Grant owner access to all users in serve mode via None ACL
- Add benchmark reports from multi-relay comparison run
- Update CLAUDE.md with binary tag handling documentation
- Bump version to v0.30.2
2025-11-26 09:52:29 +00:00
f1ddad3318 fix policy logic error caused by interface breach
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 20:46:46 +00:00
0161825be8 bump for social graph feature for neo4j v0.30.0
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 18:09:51 +00:00
655a7d9473 update workflow to update web app bundle correctly
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 15:41:01 +00:00
a03af8e05a self-detection elides self url at startup, handles multiple DNS pointers
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 13:26:37 +00:00
1522bfab2e add relay self-connection via authed pubkey
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 12:54:37 +00:00
2b8f359a83 fix workflow to fetch libsecp256k1.so
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 11:04:04 +00:00
2e865c9616 fix workflow to fetch libsecp256k1.so
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 06:03:22 +00:00
7fe1154391 fix policy load failure to panic, remove fallback case
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-25 05:49:05 +00:00
b67f7dc900 fix policy to require auth and ignore all reqs before valid auth is made
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-21 20:19:24 +00:00
ebe0012863 fix auth, read/white whitelisting and rule precedence, bump to v0.29.13
Some checks failed
Go / build-and-release (push) Has been cancelled
Policy System Verification & Testing (Latest Updates) Authentication & Security:

Verified policy system enforces authentication for all REQ and EVENT messages when enabled

Confirmed AUTH challenges are sent immediately on connection and repeated until authentication succeeds

Validated unauthenticated requests are silently rejected regardless of other policy rules

Access Control Logic:

Confirmed privileged flag only restricts read access (REQ queries), not write operations (EVENT submissions)

Validated read_allow and privileged use OR logic: users get access if EITHER they're in the allow list OR they're a party to the event (author/p-tag)
This design allows both explicit whitelisting and privacy for involved parties

Kind Whitelisting:

Verified kind filtering properly rejects unlisted events in all scenarios:

Explicit kind.whitelist: Only listed kinds accepted, even if rules exist for other kinds

Implicit whitelist (rules only): Only kinds with defined rules accepted

Blacklist mode: Blacklisted kinds rejected, others require rules

Added comprehensive test suite (10 scenarios) covering edge cases and real-world configurations
2025-11-21 16:13:34 +00:00
00a6a78a41 fix cache to disregard subscription ids
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-20 12:30:17 +00:00
d4e2f48b7e bump to v0.29.10
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-19 13:08:00 +00:00
be6cd8c740 fixed error comparing hex/binary in pubkey white/blacklist, complete neo4j and tests"
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-19 11:25:38 +00:00
8b3d03da2c fix workflow setup
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-18 20:56:18 +00:00
5bcb8d7f52 upgrade to gitea workflows
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-18 20:50:05 +00:00
b3b963ecf5 replace github workflows with gitea 2025-11-18 20:46:54 +00:00
d4fb6cbf49 fix handleevents not prompting auth for event publish with auth-required
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-18 20:26:36 +00:00
d5c0e3abfc bump to v0.29.3
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-18 18:22:39 +00:00
beed174e83 make query cache normalize filters so same query different order filters are cache hits
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-17 00:04:21 +00:00
bf8d912063 enhance spider with rate limit handling, follow list updates, and improved reconnect logic; bump version to v0.29.0
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
also reduces CPU load for spider, and minor CORS fixes
2025-11-14 21:15:24 +00:00
24eef5b5a8 fix CORS headers and a wasm experiment
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-14 19:15:50 +00:00
1d9a6903b8 bump version
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-14 12:18:01 +00:00
7169a2158f when in "none" ACL mode, privileged checks are not enforced
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-13 08:31:02 +00:00
baede6d37f extend script test to two read two write to ensure script continues running
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 15:24:58 +00:00
3e7cc01d27 make script stderr print into relay logs
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 14:41:54 +00:00
cc99fcfab5 bump to v0.27.5
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 14:38:05 +00:00
b2056b6636 bump to v0.27.5
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 13:48:23 +00:00
e9fb314496 fully test and verify policy script functionality
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 09:37:42 +00:00
597711350a fix script startup and validate with tests
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-10 12:36:55 +00:00
7113848de8 fix error handling of default policy script
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-10 11:55:42 +00:00
09bcbac20d create concurrent script runner per rule script
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
bump to v0.27.1
2025-11-10 10:56:02 +00:00
84b7c0e11c bump to v0.27.0
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-09 10:42:50 +00:00
d0dbd2e2dc implemented and tested NIP-43 invite based ACL 2025-11-09 10:41:58 +00:00
f0beb83ceb fix utf8 handling bug, bump to v0.26.4
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-08 10:29:24 +00:00
c79cd2ffee Remove deprecated files and enhance subscription stability
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Deleted obsolete files including ALL_FIXES.md, MESSAGE_QUEUE_FIX.md, PUBLISHER_FIX.md, and others to streamline the codebase.
- Implemented critical fixes for subscription stability, ensuring receiver channels are consumed and preventing drops.
- Introduced per-subscription consumer goroutines to enhance event delivery and prevent message queue overflow.
- Updated documentation to reflect changes and provide clear testing guidelines for subscription stability.
- Bumped version to v0.26.3 to signify these important updates.
2025-11-06 20:10:08 +00:00
581e0ec588 Implement comprehensive WebSocket subscription stability fixes
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Resolved critical issues causing subscriptions to drop after 30-60 seconds due to unconsumed receiver channels.
- Introduced per-subscription consumer goroutines to ensure continuous event delivery and prevent channel overflow.
- Enhanced REQ parsing to handle both wrapped and unwrapped filter arrays, eliminating EOF errors.
- Updated publisher logic to correctly send events to receiver channels, ensuring proper event delivery to subscribers.
- Added extensive documentation and testing tools to verify subscription stability and performance.
- Bumped version to v0.26.2 to reflect these significant improvements.
2025-11-06 18:21:00 +00:00
29ab350eed Implement advanced filtering capabilities in the search interface
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Added a FilterBuilder component to allow users to create complex search filters based on various criteria such as event kinds, authors, and tags.
- Introduced a FilterDisplay component to show active filters and provide an option to clear them.
- Updated the App.svelte to integrate the new filtering features, including handling filter application and clearing.
- Enhanced search functionality to utilize the new filter structure, improving the search results experience.
- Bumped version to v0.26.0 to reflect these changes.
2025-11-06 09:20:18 +00:00
88d3e3f73e Refactor authentication challenge logic in HandleCount and HandleReq
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Updated the authentication challenge conditions in both HandleCount and HandleReq functions to check for the presence of an authenticated public key.
- Introduced the schnorr package to handle public key length validation.
- Bumped version to v0.25.7 to reflect these changes.
2025-11-05 19:23:59 +00:00
36fc05b1c2 Update Go workflow to include libsecp256k1 for tests and bump version to v0.25.5
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Modified the CI workflow to copy the `libsecp256k1.so` file to the root directory for test accessibility.
- Enhanced the `InitPub` method in the `FallbackSigner` to parse the x-only public key for verification.
- Bumped version to v0.25.6 to reflect these updates.
2025-11-05 15:22:39 +00:00
ae170fc069 Update Go workflow and bump version to v0.25.4
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Removed installation steps for `libsecp256k1` from the CI workflow, streamlining the build process.
- Adjusted the build steps for release binaries to utilize pre-compiled `libsecp256k1.so` for Linux AMD64.
- Bumped version to v0.25.4 to reflect these changes.
2025-11-05 13:41:20 +00:00
7af08f9fd2 Remove deprecated test files and optimize encryption functions
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Deleted `testresults.txt` and `testmain_test.go` as they were no longer needed.
- Updated the Go workflow to streamline the build process by removing commented-out build steps for various platforms.
- Refactored encryption benchmarks to improve performance and clarity in the `benchmark_test.go` file.
- Introduced a new LICENSE file for the encryption package, specifying the MIT License.
- Enhanced the README with usage instructions and links to the NIP-44 specification.
- Bumped version to v0.25.3 to reflect these changes.
2025-11-05 13:28:17 +00:00
9d13811f6b Add test files and enhance logging in various components
- Introduced test files for the blossom and database packages to improve test coverage and ensure functionality.
- Updated logging practices by suppressing unnecessary log outputs during tests to enhance clarity and focus on relevant information.
- Refactored error handling in the `handle-message` and `handle-req` functions to avoid logging expected context cancellation errors during shutdown.
- Bumped version to v0.25.2 to reflect these updates.
2025-11-05 08:15:02 +00:00
1d12099f1c Enhance WebSocket connection management and signer implementation
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Added a pong handler to extend the read deadline upon receiving PONG messages in WebSocket connections, improving connection stability.
- Updated the signer implementation to serialize the x-only public key to 32 bytes, ensuring compatibility with the internal format.
- Refactored tests to utilize MustNew() for signer initialization, enhancing error handling during key generation.
- Bumped version to v0.25.1 to reflect these updates.
2025-11-05 06:19:28 +00:00
e0a95ca1cd Refactor signer implementation to use p8k package
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Replaced all instances of p256k1signer with the new p8k.Signer across various modules, including event creation, policy handling, and database interactions.
- Updated related test cases and benchmarks to ensure compatibility with the new signer interface.
- Bumped version to v0.25.0 to reflect these significant changes and improvements in cryptographic operations.
2025-11-04 20:05:19 +00:00
f1c636db41 Bump version to v0.24.7
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-04 12:43:18 +00:00
cefd0a98e7 Remove unused pong handler in WebSocket connection management and bump version to v0.24.6
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-04 10:44:45 +00:00