189 lines
4.6 KiB
Markdown
189 lines
4.6 KiB
Markdown
# Badger Cache Optimization Strategy
|
|
|
|
## Problem Analysis
|
|
|
|
### Initial Configuration (FAILED)
|
|
- Block cache: 2048 MB
|
|
- Index cache: 1024 MB
|
|
- **Result**: Cache hit ratio remained at 33%
|
|
|
|
### Root Cause Discovery
|
|
|
|
Badger's Ristretto cache uses a "cost" metric that doesn't directly map to bytes:
|
|
|
|
```
|
|
Average cost per key: 54,628,383 bytes = 52.10 MB
|
|
Cache size: 2048 MB
|
|
Keys that fit: ~39 keys only!
|
|
```
|
|
|
|
The cost metric appears to include:
|
|
- Uncompressed data size
|
|
- Value log references
|
|
- Table metadata
|
|
- Potentially full `BaseTableSize` (64 MB) per entry
|
|
|
|
### Why Previous Fix Didn't Work
|
|
|
|
With `BaseTableSize = 64 MB`:
|
|
- Each cache entry costs ~52 MB in the cost metric
|
|
- 2 GB cache ÷ 52 MB = ~39 entries max
|
|
- Test generates 228,000+ unique keys
|
|
- **Eviction rate: 99.99%** (everything gets evicted immediately)
|
|
|
|
## Multi-Pronged Optimization Strategy
|
|
|
|
### Approach 1: Reduce Table Sizes (IMPLEMENTED)
|
|
|
|
**Changes in `pkg/database/database.go`:**
|
|
|
|
```go
|
|
// OLD (causing high cache cost):
|
|
opts.BaseTableSize = 64 * units.Mb // 64 MB per table
|
|
opts.MemTableSize = 64 * units.Mb // 64 MB memtable
|
|
|
|
// NEW (lower cache cost):
|
|
opts.BaseTableSize = 8 * units.Mb // 8 MB per table (8x reduction)
|
|
opts.MemTableSize = 16 * units.Mb // 16 MB memtable (4x reduction)
|
|
```
|
|
|
|
**Expected Impact:**
|
|
- Cost per key should drop from ~52 MB to ~6-8 MB
|
|
- Cache can now hold ~2,000-3,000 keys instead of ~39
|
|
- **Projected hit ratio: 60-70%** (significant improvement)
|
|
|
|
### Approach 2: Enable Compression (IMPLEMENTED)
|
|
|
|
```go
|
|
// OLD:
|
|
opts.Compression = options.None
|
|
|
|
// NEW:
|
|
opts.Compression = options.ZSTD
|
|
opts.ZSTDCompressionLevel = 1 // Fast compression
|
|
```
|
|
|
|
**Expected Impact:**
|
|
- Compressed data reduces cache cost metric
|
|
- ZSTD level 1 is very fast (~500 MB/s) with ~2-3x compression
|
|
- Should reduce cost per key by another 50-60%
|
|
- **Combined with smaller tables: cost per key ~3-4 MB**
|
|
|
|
### Approach 3: Massive Cache Increase (IMPLEMENTED)
|
|
|
|
**Changes in `Dockerfile.next-orly`:**
|
|
|
|
```dockerfile
|
|
ENV ORLY_DB_BLOCK_CACHE_MB=16384 # 16 GB (was 2 GB)
|
|
ENV ORLY_DB_INDEX_CACHE_MB=4096 # 4 GB (was 1 GB)
|
|
```
|
|
|
|
**Rationale:**
|
|
- With 16 GB cache and 3-4 MB cost per key: **~4,000-5,000 keys** can fit
|
|
- This should cover the working set for most benchmark tests
|
|
- **Target hit ratio: 80-90%**
|
|
|
|
## Combined Effect Calculation
|
|
|
|
### Before Optimization:
|
|
- Table size: 64 MB
|
|
- Cost per key: ~52 MB
|
|
- Cache: 2 GB
|
|
- Keys in cache: ~39
|
|
- Hit ratio: 33%
|
|
|
|
### After Optimization:
|
|
- Table size: 8 MB (8x smaller)
|
|
- Compression: ZSTD (~3x reduction)
|
|
- Effective cost per key: ~2-3 MB (17-25x reduction!)
|
|
- Cache: 16 GB (8x larger)
|
|
- Keys in cache: **~5,000-8,000** (128-205x improvement)
|
|
- **Projected hit ratio: 85-95%**
|
|
|
|
## Trade-offs
|
|
|
|
### Smaller Tables
|
|
**Pros:**
|
|
- Lower cache cost
|
|
- Faster individual compactions
|
|
- Better cache efficiency
|
|
|
|
**Cons:**
|
|
- More files to manage (mitigated by faster compaction)
|
|
- Slightly more compaction overhead
|
|
|
|
**Verdict:** Worth it for 25x cache efficiency improvement
|
|
|
|
### Compression
|
|
**Pros:**
|
|
- Reduces cache cost
|
|
- Reduces disk space
|
|
- ZSTD level 1 is very fast
|
|
|
|
**Cons:**
|
|
- ~5-10% CPU overhead for compression
|
|
- ~3-5% CPU overhead for decompression
|
|
|
|
**Verdict:** Minor CPU cost for major cache gains
|
|
|
|
### Large Cache
|
|
**Pros:**
|
|
- High hit ratio
|
|
- Lower latency
|
|
- Better throughput
|
|
|
|
**Cons:**
|
|
- 20 GB memory usage (16 GB block + 4 GB index)
|
|
- May not be suitable for resource-constrained environments
|
|
|
|
**Verdict:** Acceptable for high-performance relay deployments
|
|
|
|
## Alternative Configurations
|
|
|
|
### For 8 GB RAM Systems:
|
|
```dockerfile
|
|
ENV ORLY_DB_BLOCK_CACHE_MB=6144 # 6 GB
|
|
ENV ORLY_DB_INDEX_CACHE_MB=1536 # 1.5 GB
|
|
```
|
|
With optimized tables+compression: ~2,000-3,000 keys, 70-80% hit ratio
|
|
|
|
### For 4 GB RAM Systems:
|
|
```dockerfile
|
|
ENV ORLY_DB_BLOCK_CACHE_MB=2560 # 2.5 GB
|
|
ENV ORLY_DB_INDEX_CACHE_MB=512 # 512 MB
|
|
```
|
|
With optimized tables+compression: ~800-1,200 keys, 50-60% hit ratio
|
|
|
|
## Testing & Validation
|
|
|
|
To test these changes:
|
|
|
|
```bash
|
|
cd /home/mleku/src/next.orly.dev/cmd/benchmark
|
|
|
|
# Rebuild with new code changes
|
|
docker compose build next-orly
|
|
|
|
# Run benchmark
|
|
sudo rm -rf data/
|
|
./run-benchmark-orly-only.sh
|
|
```
|
|
|
|
### Metrics to Monitor:
|
|
1. **Cache hit ratio** (target: >85%)
|
|
2. **Cache life expectancy** (target: >30 seconds)
|
|
3. **Average latency** (target: <3ms)
|
|
4. **P95 latency** (target: <10ms)
|
|
5. **Burst pattern performance** (target: match khatru-sqlite)
|
|
|
|
## Expected Results
|
|
|
|
### Burst Pattern Test:
|
|
- **Before**: 9.35ms avg, 34.48ms P95
|
|
- **After**: <4ms avg, <10ms P95 (60-70% improvement)
|
|
|
|
### Overall Performance:
|
|
- Match or exceed khatru-sqlite and khatru-badger
|
|
- Eliminate cache warnings
|
|
- Stable performance across test rounds
|