Add BBolt rate limiting and tune Badger defaults for large archives (v0.48.12)
Some checks failed
Go / build-and-release (push) Has been cancelled

- Increase Badger cache defaults: block 512→1024MB, index 256→512MB
- Increase serial cache defaults: pubkeys 100k→250k, event IDs 500k→1M
- Change ZSTD default from level 1 (fast) to level 3 (balanced)
- Add memory-only rate limiter for BBolt backend
- Add BBolt to database backend docs with scaling recommendations
- Document migration between Badger and BBolt backends

Files modified:
- app/config/config.go: Tuned defaults for large-scale deployments
- main.go: Add BBolt rate limiter support
- pkg/ratelimit/factory.go: Add NewMemoryOnlyLimiter factory
- pkg/ratelimit/memory_monitor.go: New memory-only load monitor
- CLAUDE.md: Add BBolt docs and scaling guide

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
woikos
2026-01-10 11:55:07 +01:00
parent 61f6027a64
commit be72b694eb
6 changed files with 286 additions and 11 deletions

View File

@@ -54,3 +54,11 @@ func MonitorFromNeo4jDriver(
) loadmonitor.Monitor {
return NewNeo4jMonitor(driver, querySem, maxConcurrency, 100*time.Millisecond)
}
// NewMemoryOnlyLimiter creates a rate limiter that only monitors process memory.
// Useful for database backends that don't have their own load metrics (e.g., BBolt).
// Since BBolt uses memory-mapped IO, memory pressure is still relevant.
func NewMemoryOnlyLimiter(config Config) *Limiter {
monitor := NewMemoryMonitor(100 * time.Millisecond)
return NewLimiter(config, monitor)
}