Commit Graph

27 Commits

Author SHA1 Message Date
f22bf3f388 Add Neo4j memory tuning config and query result limits (v0.43.0)
Some checks failed
Go / build-and-release (push) Has been cancelled
- Add Neo4j driver config options for memory management:
  - ORLY_NEO4J_MAX_CONN_POOL (default: 25) - connection pool size
  - ORLY_NEO4J_FETCH_SIZE (default: 1000) - records per batch
  - ORLY_NEO4J_MAX_TX_RETRY_SEC (default: 30) - transaction retry timeout
  - ORLY_NEO4J_QUERY_RESULT_LIMIT (default: 10000) - max results per query
- Apply driver settings when creating Neo4j connection (pool size, fetch size, retry time)
- Enforce query result limit as safety cap on all Cypher queries
- Fix QueryForSerials and QueryForIds to preserve LIMIT clauses
- Add comprehensive memory tuning documentation with sizing guidelines
- Add NIP-46 signer-based authentication for bunker connections
- Update go.mod with new dependencies

Files modified:
- app/config/config.go: Add Neo4j driver tuning config vars
- main.go: Pass new config values to database factory
- pkg/database/factory.go: Add Neo4j tuning fields to DatabaseConfig
- pkg/database/factory_wasm.go: Mirror factory.go changes for WASM
- pkg/neo4j/neo4j.go: Apply driver config, add getter methods
- pkg/neo4j/query-events.go: Enforce query result limit, fix LIMIT preservation
- docs/NEO4J_BACKEND.md: Add Memory Tuning section, update Docker example
- CLAUDE.md: Add Neo4j memory tuning quick reference
- app/handle-req.go: NIP-46 signer authentication
- app/publisher.go: HasActiveNIP46Signer check
- pkg/protocol/publish/publisher.go: NIP46SignerChecker interface
- go.mod: Add dependencies

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 02:18:05 +02:00
06063750e7 Add fixed-size type support for IdPkTs and EventRef
- Update nostr dependency to v1.0.11 with new types package
- Add IDFixed(), PubFixed(), IDHex(), PubHex() methods to IdPkTs
- Add EventRef type: 80-byte stack-allocated event reference
- Add ToEventRef()/ToIdPkTs() conversion methods
- Update tests to use IDHex() instead of hex.Enc(r.Id)

EventRef provides:
- Copy-on-assignment semantics (arrays vs slices)
- Zero heap allocations for event reference passing
- Type-safe fixed-size fields (EventID, Pubkey)

Files modified:
- go.mod, go.sum: Update nostr to v1.0.11
- pkg/interfaces/store/store_interface.go: Add methods and EventRef type
- pkg/interfaces/store/store_interface_test.go: New test file
- pkg/database/binary_tag_filter_test.go: Use IDHex()
- pkg/neo4j/fetch-event_test.go: Use IDHex(), PubHex()

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 14:47:50 +01:00
9d6280eab1 Fix duplicate REPORTS relationships in Neo4j backend (v0.36.1)
Some checks failed
Go / build-and-release (push) Has been cancelled
- Change processReport() to use MERGE instead of CREATE for REPORTS
  relationships, deduplicating by (reporter, reported, report_type)
- Add ON CREATE/ON MATCH clauses to preserve newest event data while
  preventing duplicate relationships
- Add getExistingReportEvent() helper to check for existing reports
- Add markReportEventSuperseded() to track superseded events
- Add v4 migration migrateDeduplicateReports() to clean up existing
  duplicate REPORTS relationships in databases
- Add comprehensive tests: TestReportDeduplication with subtests for
  deduplication, different types, and superseded event tracking
- Update WOT_SPEC.md with REPORTS deduplication behavior and correct
  property names (report_type, created_at, created_by_event)
- Bump version to v0.36.1

Fixes: https://git.nostrdev.com/mleku/next.orly.dev/issues/16

Files modified:
- pkg/neo4j/social-event-processor.go: MERGE-based deduplication
- pkg/neo4j/migrations.go: v4 migration for duplicate cleanup
- pkg/neo4j/social-event-processor_test.go: Deduplication tests
- pkg/neo4j/WOT_SPEC.md: Updated REPORTS documentation
- pkg/version/version: Bump to v0.36.1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 10:13:15 +01:00
96bdf5cba2 Implement Tag-based e/p model for Neo4j backend (v0.36.0)
Some checks failed
Go / build-and-release (push) Has been cancelled
- Add unified Tag-based model where e/p tags create intermediate Tag nodes
  with REFERENCES relationships to Event/NostrUser nodes
- Update save-event.go: addPTagsInBatches and addETagsInBatches now create
  Tag nodes with TAGGED_WITH and REFERENCES relationships
- Update delete.go: CheckForDeleted uses Tag traversal for kind 5 detection
- Add v3 migration in migrations.go to convert existing direct REFERENCES
  and MENTIONS relationships to the new Tag-based model
- Create comprehensive test file tag_model_test.go with 15+ test functions
  covering Tag model, filter queries, migrations, and deletion detection
- Update save-event_test.go to verify new Tag-based relationship patterns
- Update WOT_SPEC.md with Tag-Based References documentation section
- Update CLAUDE.md and README.md with Neo4j Tag-based model documentation
- Bump version to v0.36.0

This change enables #e and #p filter queries to work correctly by storing
all tags (including e/p) through intermediate Tag nodes.

Files modified:
- pkg/neo4j/save-event.go: Tag-based e/p relationship creation
- pkg/neo4j/delete.go: Tag traversal for deletion detection
- pkg/neo4j/migrations.go: v3 migration for existing data
- pkg/neo4j/tag_model_test.go: New comprehensive test file
- pkg/neo4j/save-event_test.go: Updated for new model
- pkg/neo4j/WOT_SPEC.md: Tag-Based References documentation
- pkg/neo4j/README.md: Architecture and example queries
- CLAUDE.md: Repository documentation update
- pkg/version/version: Bump to v0.36.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 09:22:05 +01:00
f16ab3077f Interim release: documentation updates and rate limiting improvements
- Add applesauce library reference documentation
- Add rate limiting test report for Badger
- Add memory monitoring for rate limiter (platform-specific implementations)
- Enhance PID-controlled adaptive rate limiting
- Update Neo4j and Badger monitors with improved load metrics
- Add docker-compose configuration
- Update README and configuration options

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 08:47:25 +01:00
ba84e12ea9 Add _graph extension support to Neo4j driver
Some checks failed
Go / build-and-release (push) Has been cancelled
- Implement TraverseFollows using Cypher path queries on FOLLOWS relationships
- Implement TraverseFollowers using reverse path traversal
- Implement FindMentions using MENTIONS relationships from p-tags
- Implement TraverseThread using REFERENCES relationships from e-tags
  with bidirectional traversal (inbound replies, outbound parents)
- Add GraphAdapter to bridge Neo4j to graph.GraphDatabase interface
- Add GraphResult type implementing graph.GraphResultI for Neo4j
- Initialize graph executor for Neo4j backend in app/main.go

The implementation uses existing Neo4j schema and relationships created
by SaveEvent() - no schema changes required. The _graph extension now
works transparently with either Badger or Neo4j backends.

Bump version to v0.35.0

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 07:07:31 +01:00
95271cbc81 Add Neo4j integration tests and query rate-limiting logic
Some checks failed
Go / build-and-release (push) Has been cancelled
Introduce comprehensive integration tests for Neo4j bug fixes covering batching, event relationships, and processing logic. Add rate-limiting to Neo4j queries using semaphores and retry policies to prevent authentication rate limiting and connection exhaustion, ensuring system stability under load.
2025-12-07 00:07:25 +00:00
1e9c447fe6 Refactor Neo4j tests and improve tag handling in Cypher
Replaces outdated Neo4j test setup with a robust TestMain, shared test database, and utility functions for test data and migrations. Improves Cypher generation for processing e-tags, p-tags, and other tags to ensure compliance with Neo4j syntax. Added integration test script and updated benchmark reports for Badger backend.
2025-12-04 20:09:24 +00:00
54ead81791 merge authors/nostruser in neo4j, add compact pubkey/e/p serial refs
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-12-03 20:49:49 +00:00
52189633d9 Unify NostrUser and Author nodes; add migrations support
Some checks failed
Go / build-and-release (push) Has been cancelled
Merged 'Author' nodes into 'NostrUser' for unified identity tracking and social graph representation. Introduced migrations framework to handle schema changes, including retroactive updates for existing relationships and constraints. Updated tests, schema definitions, and documentation to reflect these changes.
2025-12-03 20:02:41 +00:00
59247400dc Remove Dgraph support from the codebase.
Some checks failed
Go / build-and-release (push) Has been cancelled
Dgraph-related functionality, configuration, and benchmarks have been removed from the project. This streamlines the codebase to focus on supported backends, specifically eliminating Dgraph references in favor of Neo4j and other implementations. Version bumped to reflect the changes.
2025-12-03 19:33:37 +00:00
880772cab1 Remove Dgraph, check hex field case, reject if any uppercase
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-12-03 16:26:07 +00:00
1851ba39fa fix type and nil panic errors
Some checks failed
Go / build-and-release (push) Has been cancelled
1. Added Err() method to CollectedResult (pkg/neo4j/neo4j.go:68-72):
    - The resultiter.Neo4jResultIterator interface requires Err() error
    - CollectedResult was missing this method, causing the type assertion to fail
    - Since CollectedResult pre-fetches all records, Err() always returns nil
  2. Fixed nil pointer dereference in buildCypherQuery (pkg/neo4j/query-events.go:173):
    - Changed if *f.Limit > 0 to if f.Limit != nil && *f.Limit > 0
    - This prevents a panic when filters don't specify a limit
  3. Simplified parseEventsFromResult signature (pkg/neo4j/query-events.go:185):
    - Changed from func (n *N) parseEventsFromResult(result any) to accept *CollectedResult directly
    - This eliminates the runtime type assertion since ExecuteRead already returns *CollectedResult
    - Removed the now-unused resultiter import
2025-12-03 12:59:23 +00:00
0a61f274d5 implement wasm/js specific database engine 2025-12-03 12:31:25 +00:00
c8fac06f24 lint and correct cypher query code 2025-12-03 10:42:32 +00:00
69e2c873d8 Refactor for interface clarity and dependency isolation.
Some checks failed
Go / build-and-release (push) Has been cancelled
Replaced inline interface literals with dedicated, documented interface definitions in `pkg/interfaces/`. Introduced `TimeoutError`, `PolicyChecker`, and `Neo4jResultIterator` interfaces to clarify design, improve maintainability, and resolve potential circular dependencies. Updated config and constant usage rules for consistency. Incremented version to v0.31.11.
2025-12-03 06:04:50 +00:00
6c7d55ff7e Update version and add comprehensive Cypher query tests
Some checks failed
Go / build-and-release (push) Has been cancelled
Bumped version to v0.31.10. Added extensive unit and integration tests for Cypher query generation in Neo4j, including validation of WITH clause fixes and handling optional matches for various event tagging scenarios. Ensures robust handling of references, relationships, and syntactical correctness.
2025-12-02 19:29:52 +00:00
feae79af1a fix bug in cypher code that breaks queries
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-12-02 19:10:50 +00:00
016e97925a Refactor database configuration to use centralized struct
Some checks failed
Go / build-and-release (push) Has been cancelled
Replaced individual environment variable access with a unified `DatabaseConfig` struct for all database backends. This centralizes configuration management, reduces redundant code, and ensures all options are documented in `app/config/config.go`. Backward compatibility is maintained with default values and retained constructors.
2025-12-02 13:30:50 +00:00
2026591c42 update schema and add doc on updating schema 2025-11-28 06:27:46 +00:00
1810c8bef3 Fix binary tag value handling for e/p tags across database layer
Some checks failed
Go / build-and-release (push) Has been cancelled
- Update nostr library to v1.0.3 with improved binary tag support
- Replace tag.Value() calls with tag.ValueHex() to handle both binary and hex formats
- Add NormalizeTagValueForHash() for consistent filter tag normalization
- Update QueryPTagGraph to handle binary-encoded and hex-encoded pubkeys
- Fix tag matching in query-events.go using TagValuesMatchUsingTagMethods
- Add filter_utils.go with tag normalization helper functions
- Update delete operations in process-delete.go and neo4j/delete.go
- Fix ACL follows extraction to use ValueHex() for consistent decoding
- Add binary_tag_filter_test.go for testing tag value normalization
- Bump version to v0.30.3
2025-11-26 21:16:46 +00:00
6412edeabb implement preliminary implementation of graph data model 2025-11-25 18:08:44 +00:00
1c376e6e8d migrate to new nostr library 2025-11-23 08:15:06 +00:00
00a6a78a41 fix cache to disregard subscription ids
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-20 12:30:17 +00:00
be6cd8c740 fixed error comparing hex/binary in pubkey white/blacklist, complete neo4j and tests"
Some checks failed
Go / build-and-release (push) Has been cancelled
2025-11-19 11:25:38 +00:00
d4fb6cbf49 fix handleevents not prompting auth for event publish with auth-required
Some checks failed
Go / build (push) Has been cancelled
Go / release (push) Has been cancelled
2025-11-18 20:26:36 +00:00
86481a42e8 initial draft of neo4j database driver 2025-11-17 08:19:44 +00:00