Commit Graph

60 Commits

Author SHA1 Message Date
b761a04422 fix a lot of tests
also a couple disable because they are weird
2025-08-07 22:39:18 +01:00
8adb129fbe Update relay description and fix indentation in handleRelayinfo.go
- pkg/version/version.go
  - Updated `Description` to include the URL `https://orly.dev`.

- pkg/app/relay/handleRelayinfo.go
  - Fixed indentation for `Nips`, `Software`, and `Version` fields in the relay info response structure.
2025-08-07 11:06:41 +01:00
fd698af1ca Update config defaults and reorder imports in config.go
- pkg/app/config/config.go
  - Reordered imports to group and organize external and internal packages.
  - Updated the default value of `AppName` from "orly" to "ORLY".
2025-08-07 11:03:59 +01:00
ac4fd506e5 Update relay handling and bump version to v0.5.2
- .gitignore
  - Added `.idea/.name` to ignore file.

- pkg/version/version
  - Updated version from v0.5.1 to v0.5.2.

- pkg/app/relay/handleRelayinfo.go
  - Enabled `relayinfo.ProtectedEvents` in the supported NIPs.

- pkg/app/relay/spider-fetch.go
  - Added import for `orly.dev/pkg/utils/values`.
  - Updated logic to set `l` using `values.ToUintPointer(512)`.
2025-08-07 10:54:45 +01:00
b351d0fb78 fix bugs in tag comparison code
nwc walletcli now works!

bumped to v0.5.0 because NWC client now in and available
2025-08-07 09:32:53 +01:00
9c8ff2976d backporting relay client and pool from latest go-nostr 2025-08-06 22:18:26 +01:00
e57169eeae add blacklist and add to accept-event.go; Bump version to v0.4.15 2025-08-05 23:15:13 +01:00
2d0ebfe032 Merge remote-tracking branch 'upstream/main' into kwsantiago/1-public-relay-with-blacklist 2025-08-05 14:15:02 -04:00
b7b7dc7353 feat: Add blacklist support for public relays 2025-08-05 14:09:01 -04:00
1e6ce84e26 update spider seeds, set spider to fetch in spider frequency time window 2025-08-05 06:36:19 +01:00
9314467f55 bump to v0.4.13 and Enable Second Degree Follows Spidering for Follows if directory is on
Files Changed:
- pkg/version/version
    - Updated the version number from v0.4.12 to v0.4.13
- pkg/app/relay/spider.go
    - Enabled second degree of follows spidering for directory events by adding kind.FollowList if `SpiderType` is 'directory' and `SpiderSecondDegree` is not set
2025-08-05 05:58:42 +01:00
19e6520587 Bump Version: Update to v0.4.12 and Enable Second Degree Follows Spidering
Files Changed:
- pkg/version/version
    - Updated the version number from v0.4.11 to v0.4.12
- pkg/app/relay/spider.go
    - Enabled second degree of follows spidering for directory events if `SpiderType` is 'directory' and `SpiderSecondDegree` is not set
    - Added kind.MuteList to the kinds being spidered in second degree follows mode
2025-08-05 05:24:40 +01:00
9e59a6c315 Version update to 0.4.11 and enable spidering of muted relays in follows mode
Files changed:

- pkg/version/version
    - Updated the version number from v0.4.10 to v0.4.11
- pkg/app/relay/spider.go
    - Added kind.MuteList to the kinds being spidered in follows mode for non-directory events
2025-08-05 05:10:28 +01:00
9449435c65 Update configuration and add spidering feature
Files changed:

- pkg/app/config/config.go
    - Added new field `SpiderSecondDegree` for enabling second degree of follows spidering
- pkg/app/relay/spider.go
    - Modified the logic to enable spidering the second degree of follows for non-directory events if `ORLY_SPIDER_TYPE` is set to 'follows' or 'directory', and `SpiderSecondDegree` is set to true
- pkg/version/version
    - Updated version number from v0.4.8 to v0.4.10
2025-08-05 04:58:52 +01:00
df8e66d9a7 Implement spidering improvements
- pkg/app/relay/server.go
  - Modified the time ticker used for spidering with a custom duration value
  - Added a log message to indicate when the spider is running
- pkg/app/config/config.go
  - Added a new configuration parameter `SpiderTime` of type `time.Duration` to specify how often the spider should run
2025-08-05 04:39:48 +01:00
c0bd7d8da3 skip init of relay secret key if there isn't one 2025-08-04 21:27:26 +01:00
deb56664e2 Update spider seeds and bump version to v0.4.6
- pkg/app/config/config.go
  - Updated default SpiderSeeds list with new relay URL
- pkg/version/version
  - Bumped version number from v0.4.5 to v0.4.6
2025-08-02 14:03:15 +01:00
39957c2ebf implement correct handling when saving events to search for owner/peer deletes 2025-08-02 11:43:33 +01:00
7b19db5806 Remove redundant log statements and imports from spider-fetch.go
- pkg/app/relay/spider-fetch.go
  - Removed import of "orly.dev/pkg/utils/lol"
  - Removed log.I.S(ev) statement
  - Removed log.T.C(...) and log.I.F("saved event: %0x", ev.ID) statements
  - Removed redundant comments and whitespace
2025-08-01 09:35:06 +01:00
14d4417aec Bump version and add spider type configuration
pkg/version/version
- Updated version number from v0.4.0 to v0.4.1

pkg/app/config/config.go
- Added new config field `SpiderType` with default value "directory"

pkg/app/relay/peers.go
- Added check to skip empty addresses before processing peer information

pkg/app/relay/spider.go
- Modified spider fetch logic to conditionally execute based on spider type
- Added support for different kinds of events based on spider type
2025-08-01 08:52:22 +01:00
bdda37732c Remove redundant log statements from addEvent.go
- pkg/app/relay/addEvent.go
  - Removed log.I.S(pubkeys) statement
  - Removed log.I.F("sending to replica %s", a) statement
2025-07-31 22:37:34 +01:00
699ba0554e Add event logging and replica handling improvements
- pkg/app/relay/server-publish.go
  - Added log statement for saved events
- pkg/app/relay/addEvent.go
  - Added support for multiple pubkeys
  - Modified logic to skip sending events back to replicas that already received them
  - Added header with pubkeys to prevent unnecessary resending
- pkg/protocol/openapi/event.go
  - Added parsing of X-Pubkeys header to avoid resending events
  - Updated AddEvent call to use pubkeys parameter
- pkg/protocol/httpauth/nip98auth.go
  - Removed log statement for nip-98 http auth event
- pkg/interfaces/server/server.go
  - Updated AddEvent method signature to accept pubkeys instead of pubkey
- pkg/protocol/httpauth/validate.go
  - Removed log statement for tolerance value
2025-07-31 22:08:25 +01:00
c62d685fa4 implement cluster replication
todo: need to add chain of senders in a header to a header to prevent unnecessary sends
2025-07-31 15:55:07 +01:00
6935575654 reduce the batch size to avoid missing things 2025-07-30 16:08:52 +01:00
80043b46b3 make the spider fetch everything of the last hour from all followeds 2025-07-30 16:05:55 +01:00
c68654dccc Fix documentation and code formatting issues
- cmd/lerproxy/README.md
  - Fixed grammar and punctuation in note about certificate selection
  - Improved clarity in instructions for appending intermediate certificates
  - Corrected wording in explanation of CLI tool issues with certificates

- cmd/lerproxy/app/reverse.go
  - Split long line for X-Forwarded-Host header comment to improve readability

- pkg/app/relay/server-publish.go
  - Reformatted comment block for function description to fit within line length limits
  - Added comments explaining why certain events aren't deleted from the database
  - don't delete any kind of directory events

- pkg/protocol/socketapi/handleReq.go
  - Split long lines for better readability in error message and log statements
  - Improved formatting of the notice envelope message

- pkg/app/config/config.go
  - Added new configuration fields for relay cluster replication authentication

- pkg/app/relay/publish/publisher.go
  - Removed redundant package imports and logging statements
2025-07-30 15:45:47 +01:00
c91a283520 implement fully working listener/subscribe/unsubscribe publisher 2025-07-28 05:05:56 +01:00
0d7943be89 fixed bug with client message buffer causing overwriting of tag slices 2025-07-27 11:46:04 +01:00
5237fb1a1f Add notice envelope handling for relay access control
- pkg/protocol/socketapi/handleReq.go
  - Added import of "orly.dev/pkg/encoders/bech32encoding" and "orly.dev/pkg/encoders/envelopes/noticeenvelope"
  - Added logic to send a notice envelope when public read access is not allowed, listing owners' npub keys
- pkg/protocol/socketapi/handleEvent.go
  - Added import of "orly.dev/pkg/encoders/bech32encoding" and "orly.dev/pkg/encoders/envelopes/noticeenvelope"
  - Added logic to send a notice envelope when write access is not allowed, listing owners' npub keys
- pkg/version/version
  - Bumped version from v0.2.18 to v0.2.19
- pkg/interfaces/server/server.go
  - Added `OwnersPubkeys` method to return owners' public keys
- pkg/app/relay/owners-pubkeys.go
  - Created new file with implementation of `OwnersPubkeys` method for relay server
2025-07-24 09:31:11 +01:00
fdb9e18b03 Add auth handling for relay and socket API
- pkg/app/relay/handleRelayinfo.go
  - Removed logging of info variable
- pkg/protocol/socketapi/socketapi.go
  - Removed import of "orly.dev/pkg/encoders/envelopes/authenvelope"
  - Removed authentication logic including challenge sending and logging
- pkg/app/relay/auth.go
  - Removed import of "orly.dev/pkg/utils/lol"
  - Removed tracing logic around ServiceURL function
- pkg/protocol/socketapi/handleReq.go
  - Added logging for auth status
  - Added logic to send authentication challenge if required and client is not authenticated
- pkg/protocol/socketapi/handleEvent.go
  - Added import of "orly.dev/pkg/encoders/envelopes/authenvelope" and "orly.dev/pkg/encoders/reason"
  - Added logging for auth status
  - Added logic to send authentication challenge if required and client is not authenticated
- pkg/version/version
  - Bumped version from v0.2.16 to v0.2.17
2025-07-24 00:04:26 +01:00
bfa38822e0 add event to openapi, properly implement relay accept event policy and auth with owners set 2025-07-23 23:10:39 +01:00
eac5e05e77 Add noVerify flag to SaveEvent to prevent duplicate events and enhance test reliability
- pkg/database/save-event.go
  - Added `noVerify` parameter to `SaveEvent` function
  - Added check for existing event using `GetSerialById` when `noVerify` is false
  - Modified logic to handle event verification based on `noVerify` flag
- pkg/app/relay/server-publish.go
  - Added `false` as third argument to `SaveEvent` calls
- pkg/database/export_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-for-tags_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-for-kinds-authors_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-for-kinds-tags_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-for-serials_test.go
  - Added `false` as third argument to `SaveEvent` call
- main.go
  - Modified pprof handling to support different profiling types (cpu, memory, allocation)
  - Changed `Pprof` configuration from boolean to string with enum values
- pkg/app/config/config.go
  - Changed `Pprof` field type from `bool` to `string` with enum values
- pkg/database/query-for-kinds-authors-tags_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/version/version
  - Bumped version from v0.2.12 to v0.2.13
- pkg/database/fetch-event-by-serial_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-for-kinds_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/get-serials-by-range_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/database/query-events-multiple-param-replaceable_test.go
  - Added `false` as third argument to `SaveEvent` calls
- pkg/database/query-events_test.go
  - Added `false` as third argument to `SaveEvent` calls
- pkg/interfaces/store/store_interface.go
  - Updated `Saver` interface to include `noVerify` parameter in `SaveEvent` method
  - Added `SerialByIder` interface with `GetSerialById` method
- pkg/database/save-event_test.go
  - Added `false` as third argument to `SaveEvent` calls
  - Added new test case for saving existing event
- pkg/database/query-for-ids_test.go
  - Added `false` as third argument to `SaveEvent` call
- pkg/protocol/ws/client.go
  - Changed comment about context cancellation from "context is canceled" to "context is cancelled"
- pkg/app/relay/spider-fetch.go
  - Added signature checker for WebSocket connections
  - Modified logic to check for existing events before saving
  - Added logging and memory optimization improvements
2025-07-23 13:51:02 +01:00
af04f89df8 Introduce ServeMux and OpenAPI export endpoint 2025-07-23 04:10:50 +01:00
fab2f104ff Refactor SpiderFetch to use IdPkTs with Kind and optimize memory usage 2025-07-23 01:49:46 +01:00
06940efcec make spider single threaded 2025-07-23 01:11:13 +01:00
0ba36a3f67 Add memory optimization by freeing OS memory after event processing 2025-07-23 00:59:25 +01:00
d4bee83992 Add check for value log GC and sync storage after event processing 2025-07-23 00:56:58 +01:00
aabb536d13 Refactor event handling to use IdPkTs for memory optimization and improve query efficiency. 2025-07-23 00:48:32 +01:00
4f1d48c247 Refactor privilege check logs, enhance export functionality, and add new test coverage
- Updated privilege check log messages in `publisher.go` and `handleReq.go` for better clarity and consistency
- Improved event serialization by replacing `WriteTo` with `Serialize` in `export.go`
- Added unit tests for `Export` functionality in `export_test.go`
- Introduced tests for JSON tag handling in `json_tags_test.go`
- Simplified condition in `spider-fetch.go` by removing redundant checks
2025-07-22 12:47:03 +01:00
651791aec1 Add export functionality and fix privilege checks
- Added `Export` method in `database/database.go` to export events to an io.Writer
- Implemented detailed logic for exporting all or specific pubkeys' events
- Removed placeholder `Export` function with TODO comment from `database/database.go`
- Updated error handling in `handleReq.go` and `publisher.go` by using `err != nil` instead of `chk.E(err)`
- Added more detailed logging in privilege check conditions in both `publisher.go` and `handleReq.go`
- Introduced new imports such as `"fmt"` in `connection.go` for improved error message formatting
- Created a new file `export.go` under the `database` package with complete implementation of export functionality
2025-07-22 11:34:57 +01:00
b5448f4153 Add remote address to log and fix NIP20 prefix handling
- Updated `handleMessage.go` to include the real remote address in the log message when a message is received
- Removed `regexp` import and `NIP20prefixmatcher` variable from `publisher.go`
- Modified `server.go` to remove an unused parameter from the `AddEvent` method
- Added `NIP20prefixmatcher` variable and used it for checking error messages in `addEvent.go`
2025-07-21 13:35:10 +01:00
53e8e160dd Fix check-privilege logic and remove redundant condition
- Removed unused `authedIsAuthor` variable and simplified privilege check logic in `check-privilege.go`
- Replaced conditional return based on `authedIsAuthor` with direct use of `privileged` flag
- Simplified the logic for checking if authed pubkey is mentioned in event tags
2025-07-21 11:36:57 +01:00
90c9198ebe Fix pluralization and add background fetch of metadata and relay lists
- Updated log message to use correct plurals for `owners`, `pubkey`, and related variables
- Added background fetching of profile metadata, relay list metadata, and DM relays list using `SpiderFetch` in the goroutine
- Modified `server.go` to import `"orly.dev/pkg/protocol/socketapi"` correctly and initialize `listeners` with the updated constructor
2025-07-20 23:57:41 +01:00
4bbbbb1bb6 Add DMRelaysList.K to kind.go and improve error messages with remote address in connection.go
- Added `DMRelaysList.K` constant to `pkg/encoders/kind/kind.go`
- Removed unused `"errors"` import from `pkg/protocol/ws/connection.go`
- Updated error messages in `WriteMessage`, `ReadMessage`, and related functions in `pkg/protocol/ws/connection.go` to include the remote address for better debugging
- Changed error handling from `chk.E(err)` to `chk.T(err)` in `pkg/encoders/envelopes/eventenvelope/eventenvelope.go`
- Updated ticker interval from 30 minutes to 1 hour in `pkg/app/relay/server.go`
2025-07-20 22:03:30 +01:00
56ab6eaa81 Remove redundant logging statements
- Removed log statement in `pkg/app/relay/spider-fetch.go` for batch processing
- Removed log level output in `main.go`
- Removed configuration logging in `pkg/app/config/config.go`
- Removed unused log import in `pkg/protocol/ws/subscription.go`
- Removed connection success log in `pkg/protocol/ws/client.go`
- Added early return condition for non-follow-list kind in `spider-fetch.go`
2025-07-20 21:52:04 +01:00
e3c931fcf9 Make spider fetch concurrent with mutex and waitgroup
- Added goroutines and sync primitives to handle concurrent relay connections in spider-fetch.go
- Added connection logging to client.go
2025-07-20 21:42:35 +01:00
0544159d4b added fetch of metadata and relay lists, and private flag to stop spider fetching 2025-07-20 11:42:39 +01:00
65e1dd6183 fully implemented spider to fetch follows of owners, their follows, and the owner mute list 2025-07-20 09:42:06 +01:00
0e83a56025 completed first level of owners followed pubkeys 2025-07-19 19:37:30 +01:00
681cdb3a64 Standardize parameter/return value comments in protocol/socketapi package
- Updated `pkg/protocol/socketapi/handleClose.go` to use "# Parameters" and "# Return Values" for consistent comment formatting
- Updated `pkg/protocol/socketapi/handleReq.go` with standardized parameter and return value documentation
- Modified `pkg/interfaces/relay/interface.go` to align comment style with parameter/return value sections
- Standardized comments in `pkg/protocol/socketapi/pinger.go` using "# Parameters" format
- Improved comment structure in `pkg/protocol/socketapi/socketapi.go` for parameter documentation
- Updated `pkg/protocol/socketapi/handleEvent.go` with consistent return value comment formatting
2025-07-19 14:19:54 +01:00