Commit Graph

52 Commits

Author SHA1 Message Date
1e9c447fe6 Refactor Neo4j tests and improve tag handling in Cypher
Replaces outdated Neo4j test setup with a robust TestMain, shared test database, and utility functions for test data and migrations. Improves Cypher generation for processing e-tags, p-tags, and other tags to ensure compliance with Neo4j syntax. Added integration test script and updated benchmark reports for Badger backend.
2025-12-04 20:09:24 +00:00
6b98c23606 add first draft graph query implementation
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-12-04 09:28:13 +00:00
8dbc19ee9e Add benchmark results for multiple Nostr relay backends
Introduced comprehensive benchmarks for `next-orly-badger`, `next-orly-neo4j`, and `nostr-rs-relay` backends, covering peak throughput, burst patterns, mixed read/write, query, and concurrent query/store tests. Reports include detailed performance metrics (e.g., events/sec, latency, success rates) and are saved as text and AsciiDoc formats. Aggregate summary also generated for testing consistency across relay implementations.
2025-12-04 05:43:20 +00:00
59247400dc Remove Dgraph support from the codebase.
Some checks failed
Go / build-and-release (push) Has been cancelled
Dgraph-related functionality, configuration, and benchmarks have been removed from the project. This streamlines the codebase to focus on supported backends, specifically eliminating Dgraph references in favor of Neo4j and other implementations. Version bumped to reflect the changes.
2025-12-03 19:33:37 +00:00
6bd56a30c9 remove dgraph completely
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-12-03 16:44:49 +00:00
69e2c873d8 Refactor for interface clarity and dependency isolation.
Some checks failed
Go / build-and-release (push) Has been cancelled
Replaced inline interface literals with dedicated, documented interface definitions in `pkg/interfaces/`. Introduced `TimeoutError`, `PolicyChecker`, and `Neo4jResultIterator` interfaces to clarify design, improve maintainability, and resolve potential circular dependencies. Updated config and constant usage rules for consistency. Incremented version to v0.31.11.
2025-12-03 06:04:50 +00:00
fad39ec201 Add serve mode, fix binary tags, document CLI tools, improve Docker
Some checks failed
Go / build-and-release (push) Has been cancelled
- Add 'serve' subcommand for ephemeral RAM-based relay at /dev/shm with
  open ACL mode for testing and benchmarking
- Fix e-tag and p-tag decoding to use ValueHex()/ValueBinary() methods
  instead of Value() which returns raw bytes for binary-optimized storage
- Document all command-line tools in readme.adoc (relay-tester, benchmark,
  stresstest, blossomtest, aggregator, convert, FIND, policytest, etc.)
- Switch Docker images from Alpine to Debian for proper libsecp256k1
  Schnorr signature and ECDH support required by Nostr
- Upgrade Docker Go version from 1.21 to 1.25
- Add ramdisk mode (--ramdisk) to benchmark script for eliminating disk
  I/O bottlenecks in performance measurements
- Add docker-compose.ramdisk.yml for tmpfs-based benchmark volumes
- Add test coverage for privileged policy with binary-encoded p-tags
- Fix blossom test to expect 200 OK for anonymous uploads when auth is
  not required (RequireAuth=false with ACL mode 'none')
- Update follows ACL to handle both binary and hex p-tag formats
- Grant owner access to all users in serve mode via None ACL
- Add benchmark reports from multi-relay comparison run
- Update CLAUDE.md with binary tag handling documentation
- Bump version to v0.30.2
2025-11-26 09:52:29 +00:00
da058c37c0 blossom works fully correctly 2025-11-23 12:32:53 +00:00
1c376e6e8d migrate to new nostr library 2025-11-23 08:15:06 +00:00
86cf8b2e35 unignore files that should be there 2025-11-22 20:12:55 +00:00
ef51382760 optimize e and p tags 2025-11-22 19:40:48 +00:00
55add34ac1 add rely-sqlite to benchmark
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-20 20:55:37 +00:00
1b279087a9 add vertexes between npubs and events, use for p tags 2025-11-20 09:16:54 +00:00
f89f41b8c4 full benchmark run 2025-11-19 12:22:04 +00:00
be6cd8c740 fixed error comparing hex/binary in pubkey white/blacklist, complete neo4j and tests"
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-19 11:25:38 +00:00
1d4d877a10 fix auth-required not sending immediate challenge, benchmark leak 2025-11-18 18:21:11 +00:00
038d1959ed add dgraph backend to benchmark suite with safe type assertions for multi-backend support 2025-11-17 16:52:38 +00:00
dfe8b5f8b2 add a filter query cache 512mb that stores already decoded recent query results
this should improve performance noticeably for typical kind 1 client queries
2025-11-16 18:29:53 +00:00
95bcf85ad7 optimizing badger cache, won a 10-15% improvement in most benchmarks 2025-11-16 15:07:36 +00:00
bf8d912063 enhance spider with rate limit handling, follow list updates, and improved reconnect logic; bump version to v0.29.0
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
also reduces CPU load for spider, and minor CORS fixes
2025-11-14 21:15:24 +00:00
baede6d37f extend script test to two read two write to ensure script continues running
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 15:24:58 +00:00
cc99fcfab5 bump to v0.27.5
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-11 14:38:05 +00:00
b4760c49b6 implement messages and operations for FIND 2025-11-08 08:54:58 +00:00
581e0ec588 Implement comprehensive WebSocket subscription stability fixes
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Resolved critical issues causing subscriptions to drop after 30-60 seconds due to unconsumed receiver channels.
- Introduced per-subscription consumer goroutines to ensure continuous event delivery and prevent channel overflow.
- Enhanced REQ parsing to handle both wrapped and unwrapped filter arrays, eliminating EOF errors.
- Updated publisher logic to correctly send events to receiver channels, ensuring proper event delivery to subscribers.
- Added extensive documentation and testing tools to verify subscription stability and performance.
- Bumped version to v0.26.2 to reflect these significant improvements.
2025-11-06 18:21:00 +00:00
202d3171f9 Implement multi-platform build system with pure Go support
- Introduced a comprehensive build system that supports multiple platforms (Linux, macOS, Windows, Android) using pure Go builds (`CGO_ENABLED=0`).
- Updated all build and test scripts to ensure compatibility with the new purego approach, allowing for dynamic loading of `libsecp256k1` at runtime.
- Added detailed documentation on the build process, platform detection, and deployment options.
- Enhanced CI/CD workflows to automate builds for all supported platforms and include necessary libraries in releases.
- Updated `.gitignore` to exclude build output files.
- Created new documentation files for deployment and multi-platform build summaries.
2025-11-04 20:29:19 +00:00
e0a95ca1cd Refactor signer implementation to use p8k package
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Replaced all instances of p256k1signer with the new p8k.Signer across various modules, including event creation, policy handling, and database interactions.
- Updated related test cases and benchmarks to ensure compatibility with the new signer interface.
- Bumped version to v0.25.0 to reflect these significant changes and improvements in cryptographic operations.
2025-11-04 20:05:19 +00:00
e56bf76257 Add NIP-11 relay synchronization and group management features
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced a new `sync` package for managing NIP-11 relay information and relay group configurations.
- Implemented a cache for NIP-11 documents, allowing retrieval of relay public keys and authoritative configurations.
- Enhanced the sync manager to update peer lists based on authoritative configurations from relay group events.
- Updated event handling to incorporate policy checks during event imports, ensuring compliance with relay rules.
- Refactored various components to utilize the new `sha256-simd` package for improved performance.
- Added comprehensive tests to validate the new synchronization and group management functionalities.
- Bumped version to v0.24.1 to reflect these changes.
2025-11-03 18:17:15 +00:00
2614b51068 Refactor crypto package to use p256k1 signer
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Replaced the p256k package with p256k1.mleku.dev/signer across the codebase, updating all instances where the previous signer was utilized.
- Removed the deprecated p256k package, including all related files and tests, to streamline the codebase and improve maintainability.
- Updated various components, including event handling, database interactions, and protocol implementations, to ensure compatibility with the new signer interface.
- Enhanced tests to validate the new signing functionality and ensure robustness across the application.
- Bumped version to v0.23.3 to reflect these changes.
2025-11-03 10:21:31 +00:00
3486d3d4ab added simple websocket test
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- bump to v0.21.1
2025-10-30 19:32:45 +00:00
80ab3caa5f Implement policy-based event filtering and add integration tests
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Enhanced the HandleReq function to incorporate policy checks for privileged events, ensuring only authorized users can access sensitive data.
- Introduced a new integration test suite for policy filtering, validating the behavior of event access based on user authentication and policy rules.
- Added a script to automate the policy filter integration tests, improving testing efficiency and reliability.
- Updated version to v0.20.2 to reflect the new features and improvements.
2025-10-30 17:51:15 +00:00
da1119db7c Enhance aggregator functionality for Nostr event collection
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Updated the aggregator to support both public (npub) and private (nsec) key inputs for event searching, enabling authentication for relays that require it.
- Implemented bloom filter loading and appending capabilities for efficient incremental data collection.
- Added timeout parameters for maximum runtime and stuck progress detection to improve reliability.
- Enhanced README with detailed usage instructions, authentication behavior, and examples for incremental collection.
- Bumped version to v0.17.16.
2025-10-23 13:00:01 +01:00
4c53709e2d Add aggregator functionality for Nostr event collection
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Introduced a new `aggregator` package to search for events related to a specific npub across multiple Nostr relays.
- Implemented dynamic relay discovery from relay list events and progressive backward time-based fetching for comprehensive historical data collection.
- Added a bloom filter for memory-efficient event deduplication with a low false positive rate.
- Enhanced memory management with real-time monitoring and automatic garbage collection.
- Updated README with usage instructions, features, and detailed explanations of event discovery and memory management strategies.
- Bumped version to v0.17.15.
2025-10-23 12:17:50 +01:00
95c6082564 Implement blacklisting for IPs and enhance follow list fetching
- Added functionality to handle blacklisted IPs, allowing connections to remain open until a timeout is reached.
- Introduced periodic fetching of admin follow lists to improve synchronization with relay data.
- Updated WebSocket message size limits to accommodate larger payloads.
- Enhanced logging for better traceability during follow list fetching and event processing.
- Refactored event subscription logic to improve clarity and maintainability.
2025-10-21 16:31:17 +01:00
65e8ab4fbe Enhance logging and add policy test client
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
- Refactored logOutput function to trace-log stdout and stderr lines for better debugging.
- Introduced a new policy test client in cmd/policytest/main.go to publish events and check policy responses.
- Added a script to run the policy test, ensuring proper cleanup and error handling.
- Updated policy event processing to ensure newline-terminated JSON for compatibility with shell-readers.
2025-10-20 16:30:39 +01:00
3c11aa6f01 refactored SaveEvent to return if it is replacing an event
Some checks failed
Go / build (push) Has been cancelled
2025-10-10 22:18:53 +01:00
bc5177e0ec refactor save event method to expose whether it replaced an event 2025-10-10 22:16:07 +01:00
9f39ca8a62 Refactor export functionality in App.svelte to support both GET and POST methods for event exports, enhancing flexibility in user permissions. Update server-side handling to accommodate pubkey filtering and improve response handling for file downloads. Adjust UI components to reflect these changes, ensuring a seamless user experience. 2025-10-09 14:55:29 +01:00
48c7fab795 Improve logging and handling for WebSocket message processing, delivery, and diagnostics.
Some checks failed
Go / build (push) Has been cancelled
- Enhanced logging for WebSocket writes, message handling, and delivery timing.
- Added diagnostics for slow deliveries, failures, and context timeouts.
- Incorporated extensive error handling for malformed messages and client notifications.
- Enabled command results and refined subscription management.
- Introduced detailed connection state tracking and metrics for messages, requests, and events.
- Added new `run-market-probe.sh` script for relay testing and Market seeding.
2025-10-01 08:27:22 +01:00
e58eb1d3e3 Remove commented-out debug logs and update rules for Go version and Nostr protocol documentation. 2025-09-30 13:11:41 +01:00
90f21fbcd1 Add detailed benchmark results for multiple relays.
- Included results for `relayer-basic`, `strfry`, and `nostr-rs-relay` relay benchmarks.
- Comprehensive performance metrics added for throughput, latency, query, and concurrent operations.
- Reports saved as plain text and AsciiDoc formats.
2025-09-20 12:06:57 +01:00
81a40c04e5 Refactor publishCacheEvents for concurrent publishing and optimize database access.
- Updated `publishCacheEvents` to utilize multiple concurrent connections for event publishing.
- Introduced worker-based architecture leveraging `runtime.NumCPU` for parallel uploads.
- Optimized database fetch logic in `FetchEventsBySerials` for improved maintainability and performance.
- Bumped version to `v0.4.8`.
2025-09-20 04:10:59 +01:00
58a9e83038 Refactor publishCacheEvents and publisherWorker to use fire-and-forget publishing.
- Replaced `Publish` calls with direct event envelope writes, removing wait-for-OK behavior.
- Simplified `publishCacheEvents` logic, removed per-publish timeout contexts, and updated return values.
- Adjusted log messages to reflect "sent" instead of "published."
- Enhanced relay stability with delays between successful publishes.
- Removed unused `publishTimeout` parameter from `publisherWorker` and main logic.
2025-09-20 03:48:50 +01:00
22cde96f3f Remove bufpool references and unused imports, optimize memory operations.
- Removed `bufpool` usage throughout `tag`, `tags`, and `event` packages for memory efficiency.
- Replaced in-place buffer modifications with independent, deep-copied allocations to prevent unintended mutations.
- Added new `Clone` method for deep copying `event.E`.
- Ensured valid JSON emission for nil `Tags` in `event` marshaling.
- Introduced `cmd/stresstest` for relay stress-testing with detailed workload generation and query simulation.
2025-09-19 16:17:44 +01:00
e521b788fb Delete outdated benchmark reports and results.
Removed old benchmark reports and detailed logs from the repository to clean up unnecessary files. These reports appear to be auto-generated and no longer relevant for ongoing development.
2025-09-15 05:00:19 +01:00
c45276ef08 Optimize deletion timestamp lookup by replacing sorting logic with linear scan to improve performance. Add profiling support with cmd/benchmark/profile.sh, introduce network load testing in benchmarks, and update benchmark reports with additional latency metrics (P90, bottom 10%). 2025-09-12 23:47:53 +01:00
fefa4d202e completed basic benchmark 2025-09-12 21:30:27 +01:00
110223fc4e Migrate internal module imports to unified package path.
Replaced legacy `*.orly` module imports with `next.orly.dev/pkg` paths across the codebase for consistency. Removed legacy `go.mod` files from sub-packages, consolidating dependency management. Added Dockerfiles and configurations for benchmarking environments.
2025-09-12 16:12:31 +01:00
b6ea3d5181 Add HandleDelete and GetSerialsFromFilter methods, integrate admin keys handling, and enhance constraints API. Include a new CLI convert tool for key translation. 2025-09-07 13:33:25 +01:00
5d04afd748 Remove unused eventpool package, improve logging levels, standardize websocket handling, and add HandleClose functionality. 2025-09-02 23:01:13 +01:00
91d95c6f1a Migrate package imports from next.orly.dev to new orly domain structure; add new varint and binary encoders with comprehensive tests; enhance existing tag and envelope implementations with additional methods, validations, and test coverage; introduce shared test.sh script for streamlined testing across modules. 2025-08-31 16:52:24 +01:00