implement messages and operations for FIND

This commit is contained in:
2025-11-08 09:02:32 +00:00
parent b4760c49b6
commit 5d04193bb7

View File

@@ -0,0 +1,683 @@
# Go Reference Type Simplification - Revised Proposal
## Executive Summary
Keep Go's convenient syntax (slicing, `<-`, `for range`) while making reference semantics **explicit through pointer types**. This reduces cognitive load and improves safety without sacrificing ergonomics.
## Core Principle: Explicit Pointers, Convenient Syntax
**The Key Insight:**
- Make slices/maps/channels explicitly `*[]T`, `*map[K]V`, `*chan T`
- Keep convenient operators (auto-dereference like struct methods do)
- Eliminate special allocation functions (`make()`)
- Add explicit control where it matters (grow, clone)
## Proposed Changes
### 1. Slices Become `*[]T` (Explicit Pointers)
**Current Problem:**
```go
s := []int{1, 2, 3} // Looks like value, is reference
s2 := s // Copies reference - HIDDEN SHARING
s2[0] = 99 // Mutates s too! Not obvious
```
**Proposed:**
```go
s := &[]int{1, 2, 3} // Explicit pointer allocation
s2 := s // Copies pointer - OBVIOUS SHARING
s2[0] = 99 // Mutates s too - but now obvious!
// Slicing still works (auto-dereference)
sub := s[1:3] // Returns *[]int (new slice header, same backing)
sub := s[1:3:5] // Full slicing with capacity still works
// To copy data, be explicit
s3 := s.Clone() // Deep copy
s3 := &[]int(*s) // Alternative: copy via literal
// Append works as before
s.Append(4, 5, 6) // Implicit grow if needed (fine!)
s.Grow(100) // Explicit capacity increase
```
**What Changes:**
- ✅ Allocation: `&[]T{}` instead of `make([]T, len, cap)`
- ✅ Type: `*[]int` instead of `[]int`
- ✅ Explicit clone: Must call `.Clone()` to copy data
- ✅ Explicit grow: `.Grow(n)` for pre-allocation
- ❌ Slicing syntax: **KEEP IT** - `s[i:j]` still works
- ❌ Append behavior: **KEEP IT** - implicit growth is fine
- ❌ Auto-dereference: Like methods, `s[i]` auto-derefs
**Benefits:**
- Assignment `s2 := s` is obviously pointer copy
- Function parameters `func f(s *[]int)` show mutation potential
- Still convenient: slicing and indexing work as before
### 2. Maps Become `*map[K]V` (Explicit Pointers)
**Current Problem:**
```go
m := make(map[string]int) // Special make() function
m2 := m // HIDDEN reference sharing
var m3 map[string]int // nil map
v := m3["key"] // OK - returns zero value
m3["key"] = 42 // PANIC! Nil map write trap
```
**Proposed:**
```go
m := &map[string]int{} // Explicit pointer allocation
m := &map[string]int{ // Literal initialization
"key": 42,
}
m2 := m // Obviously copies pointer
// Map operations auto-dereference
m["key"] = 42 // Auto-deref (like s[i] for slices)
v := m["key"]
v, ok := m["key"]
// Nil pointer is consistent
var m3 *map[string]int // nil pointer
v := m3["key"] // PANIC - nil pointer deref (consistent!)
m3 = &map[string]int{} // Must allocate
m3["key"] = 42 // Now OK
// Copying requires explicit clone
m4 := m.Clone() // Deep copy
```
**What Changes:**
- ✅ Allocation: `&map[K]V{}` instead of `make(map[K]V)`
- ✅ Type: `*map[K]V` instead of `map[K]V`
- ✅ Nil behavior: Consistent nil pointer panic
- ✅ Explicit clone: Must call `.Clone()`
- ❌ Map syntax: **KEEP IT** - `m[k]` auto-derefs
**Benefits:**
- Obvious pointer semantics
- No special nil-map read-only trap
- Clear when data is shared
### 3. Channels Become `*chan T` (Explicit Pointers)
**Current Problem:**
```go
ch := make(chan int, 10) // Special make() function
ch2 := ch // HIDDEN reference sharing
var ch3 chan int // nil channel
ch3 <- 42 // BLOCKS FOREVER! Silent deadlock trap
```
**Proposed:**
```go
ch := &chan int{cap: 10} // Explicit pointer allocation
ch := &chan int{} // Unbuffered (cap: 0)
ch2 := ch // Obviously copies pointer
// Channel operations auto-dereference
ch <- 42 // KEEP <- syntax!
v := <-ch
v, ok := <-ch
// for range still works
for v := range ch { // KEEP for range!
process(v)
}
// select still works
select { // KEEP select!
case v := <-ch:
handle(v)
case ch2 <- 42:
sent()
}
// Nil pointer is consistent
var ch3 *chan int // nil pointer
ch3 <- 42 // PANIC - nil pointer deref (consistent!)
// Directional channels as type aliases or interfaces
type SendOnly[T any] = *chan T // Could restrict at type level
func send(ch *chan int) {} // Or just document convention
```
**What Changes:**
- ✅ Allocation: `&chan T{cap: n}` instead of `make(chan T, n)`
- ✅ Type: `*chan T` instead of `chan T`
- ✅ Nil behavior: Consistent nil pointer panic
- ❌ Send/receive: **KEEP `<-` syntax**
- ❌ Select: **KEEP `select` statement**
- ❌ For range: **KEEP `for range ch`**
**Benefits:**
- Obvious pointer semantics
- No silent nil-channel blocking trap
- Keep all the convenient syntax
- Directional types could be interfaces if needed
### 4. Unified Allocation: Eliminate `make()`
**Before (Three Allocation Primitives):**
```go
new(T) // Returns *T (zero value)
make([]T, len, cap) // Returns []T (special)
make(map[K]V, hint) // Returns map[K]V (special)
make(chan T, buf) // Returns chan T (special)
```
**After (One Allocation Syntax):**
```go
new(T) // Returns *T (zero value)
&T{} // Returns *T (composite literal)
&[]T{} // Returns *[]T (empty slice)
&[n]T{} // Returns *[n]T (array)
&map[K]V{} // Returns *map[K]V (empty map)
&chan T{} // Returns *chan T (unbuffered)
&chan T{cap: 10} // Returns *chan T (buffered)
```
**Eliminate:**
-`make()` entirely
- ❌ Special capacity/hint parameters (use methods instead)
### 5. Type System Unification
**Before:**
```
Value types: int, float, bool, struct, [N]T
Reference types: []T, map[K]V, chan T (SPECIAL SEMANTICS)
Pointer types: *T
```
**After:**
```
Value types: int, float, bool, struct, [N]T
Pointer types: *T (including *[]T, *map[K]V, *chan T - UNIFIED)
```
All pointer types have consistent semantics:
- Assignment copies the pointer
- Nil pointer dereference panics consistently
- Auto-dereference for convenient syntax
- Explicit `.Clone()` for deep copy
## Syntax Comparison
### Slices
**Before:**
```go
// Many ways to create
var s []int // nil slice
s = []int{} // empty slice
s = make([]int, 10) // len=10, cap=10
s = make([]int, 10, 20) // len=10, cap=20
s = []int{1, 2, 3} // literal
// Slicing
sub := s[1:3] // subslice
sub = s[:3] // from start
sub = s[1:] // to end
sub = s[:] // full slice
sub = s[1:3:5] // with capacity
// Append
s = append(s, 4) // might reallocate
s = append(s, items...) // spread
// Copy (manual)
s2 := make([]int, len(s))
copy(s2, s)
```
**After:**
```go
// One way to create
var s *[]int // nil pointer
s = &[]int{} // empty slice
s = &[10]int{}[:] // len=10 from array
s = &[]int{1, 2, 3} // literal
// Slicing (UNCHANGED)
sub := s[1:3] // auto-deref, returns *[]int
sub = s[:3]
sub = s[1:]
sub = s[:]
sub = s[1:3:5]
// Append (UNCHANGED)
s.Append(4) // might reallocate (fine!)
s.Append(items...) // spread
// Explicit operations
s.Grow(100) // pre-allocate capacity
s2 := s.Clone() // explicit deep copy
```
### Maps
**Before:**
```go
// Many ways to create
var m map[K]V // nil map
m = map[K]V{} // empty map
m = make(map[K]V) // empty map
m = make(map[K]V, 100) // with hint
m = map[K]V{k: v} // literal
// Access
m[k] = v
v = m[k]
v, ok = m[k]
// Copy (manual)
m2 := make(map[K]V, len(m))
for k, v := range m {
m2[k] = v
}
```
**After:**
```go
// One way to create
var m *map[K]V // nil pointer
m = &map[K]V{} // empty map
m = &map[K]V{k: v} // literal
// Access (UNCHANGED)
m[k] = v // auto-deref
v = m[k]
v, ok = m[k]
// Explicit operations
m2 := m.Clone() // explicit deep copy
```
### Channels
**Before:**
```go
// Create
ch := make(chan int) // unbuffered
ch := make(chan int, 10) // buffered
// Operations
ch <- 42 // send
v := <-ch // receive
v, ok := <-ch // receive with closed check
close(ch)
// for range
for v := range ch {
process(v)
}
// select
select {
case v := <-ch:
handle(v)
case <-timeout:
timeout()
}
```
**After:**
```go
// Create
ch := &chan int{} // unbuffered
ch := &chan int{cap: 10} // buffered
// Operations (UNCHANGED)
ch <- 42 // auto-deref
v := <-ch
v, ok := <-ch
ch.Close() // method instead of builtin
// for range (UNCHANGED)
for v := range ch {
process(v)
}
// select (UNCHANGED)
select {
case v := <-ch:
handle(v)
case <-timeout:
timeout()
}
```
## Grammar Simplification
### Eliminated Syntax
1. **`make()` builtin** - 3 different forms → 0
- `make([]T, n, cap)``&[]T{}` + `.Grow(cap)`
- `make(map[K]V, hint)``&map[K]V{}`
- `make(chan T, buf)``&chan T{cap: buf}`
2. **Dual allocation semantics** - 2 primitives → 1
- `new(T)` and `make(T)` → just `new(T)` or `&T{}`
### Preserved Syntax
1. ✅ Slice expressions: `s[i:j]`, `s[i:j:k]`
2. ✅ Channel operators: `<-ch`, `ch<-`
3. ✅ Select statement: `select { case ... }`
4. ✅ Range over channels: `for v := range ch`
5. ✅ Map/slice indexing: `m[k]`, `s[i]`
6. ✅ Auto-dereference: Like methods on `*T`
## New Built-in Methods
### Slices (`*[]T`)
```go
s := &[]int{1, 2, 3}
// Capacity management
s.Grow(n int) // Ensure capacity for n more elements
s.Cap() int // Current capacity
s.Len() int // Current length
// Modification
s.Append(items ...T) // Append items (implicit grow OK)
s.Insert(i int, items ...T) // Insert at index
s.Delete(i, j int) // Delete s[i:j]
s.Clear() // Set length to 0
// Copying
s.Clone() *[]T // Deep copy
s.Slice(i, j int) *[]T // Alternative to s[i:j]
```
### Maps (`*map[K]V`)
```go
m := &map[string]int{}
// Capacity
m.Len() int // Number of keys
// Modification
m.Clear() // Remove all keys
m.Delete(k K) // Delete key
// Copying
m.Clone() *map[K]V // Deep copy
// Bulk operations
m.Keys() *[]K // All keys
m.Values() *[]V // All values
m.Merge(other *map[K]V) // Merge other into m
```
### Channels (`*chan T`)
```go
ch := &chan int{cap: 10}
// Metadata
ch.Len() int // Items in buffer
ch.Cap() int // Buffer capacity
// Control
ch.Close() // Close channel (method vs builtin)
```
## Auto-Dereference Rules
Like struct methods today, pointer types auto-dereference:
```go
type Person struct { name string }
func (p *Person) Name() string { return p.name }
p := &Person{name: "Alice"}
n := p.Name() // Auto-deref: (*p).Name()
// Same for new pointer types
s := &[]int{1, 2, 3}
v := s[0] // Auto-deref: (*s)[0]
sub := s[1:3] // Auto-deref: (*s)[1:3]
m := &map[K]V{}
v = m[k] // Auto-deref: (*m)[k]
ch := &chan int{}
ch <- 42 // Auto-deref: (*ch) <- 42
v = <-ch // Auto-deref: <-(*ch)
```
**Rule:** Pointer to slice/map/channel auto-derefs for indexing, slicing, and channel ops.
## Concurrency Safety
### Before: Implicit Sharing
```go
func worker(s []int, wg *sync.WaitGroup) {
defer wg.Done()
s[0] = 99 // RACE - not obvious from signature
}
s := []int{1, 2, 3}
var wg sync.WaitGroup
wg.Add(2)
go worker(s, &wg) // Sharing not obvious
go worker(s, &wg) // Two goroutines mutate same slice
wg.Wait()
```
### After: Explicit Sharing
```go
func worker(s *[]int, wg *sync.WaitGroup) {
defer wg.Done()
(*s)[0] = 99 // RACE - but obvious from *[]int
}
s := &[]int{1, 2, 3}
var wg sync.WaitGroup
wg.Add(2)
go worker(s, &wg) // OBVIOUS pointer sharing
go worker(s, &wg) // Clear that both access same data
wg.Wait()
```
**Benefits:**
- Function signature shows mutation: `func f(s *[]int)` vs `func f(s []int)`
- Pointer copy is obvious: `s2 := s` (copies pointer)
- Value copy requires explicit clone: `s2 := s.Clone()`
### Pattern: Immutable by Default
```go
// Current Go - unclear if mutation happens
func ProcessSlice(s []int) []int {
s[0] = 99 // Mutates caller's slice!
return s
}
// Proposed - explicit mutation
func ProcessSlice(s *[]int) {
(*s)[0] = 99 // Clear mutation
}
// Or value semantics (copy)
func ProcessSlice(s []int) []int { // Note: NOT pointer
result := &[]int(s) // Explicit copy from value
(*result)[0] = 99 // Mutate copy
return result
}
```
## Migration Path
### Phase 1: Allow Both (Backward Compatible)
```go
// Old style still works
s := []int{1, 2, 3}
s = append(s, 4)
// New style also works (same runtime behavior)
s := &[]int{1, 2, 3}
s.Append(4)
// Add deprecation warnings
make([]int, 10) // WARNING: Use &[]int{} or &[10]int{}[:]
```
### Phase 2: Deprecate Old Forms
```go
// Compiler warnings
[]int{1, 2, 3} // WARNING: Use &[]int{1, 2, 3}
make([]int, 10) // WARNING: Use &[]int{} with .Grow(10)
make(map[K]V) // WARNING: Use &map[K]V{}
make(chan T, 10) // WARNING: Use &chan T{cap: 10}
```
### Phase 3: Breaking Change
```go
// Only new syntax allowed
&[]int{1, 2, 3} // OK
&map[K]V{} // OK
&chan T{cap: 10} // OK
[]int{1, 2, 3} // ERROR: Use &[]int{1, 2, 3}
make([]int, 10) // ERROR: Removed
```
## Implementation Impact
### Compiler Changes
**New:**
- Auto-dereference for `*[]T`, `*map[K]V`, `*chan T`
- Built-in methods (`.Append()`, `.Clone()`, `.Grow()`, etc.)
- Composite literal fields: `&chan T{cap: 10}`
**Removed:**
- `make()` builtin (3 forms)
- Special case type checking for reference types
**Preserved:**
- Slice expressions `s[i:j:k]`
- Channel operators `<-`
- Select statement
- Range over channels
- All runtime implementations
### Runtime Changes
**Minimal:**
- Same memory layout for slices/maps/channels
- Same GC behavior
- Same scheduler
- No performance impact
**API:**
- Add runtime functions for `.Clone()`, `.Grow()`, etc.
- These can be compiler intrinsics for performance
## Complexity Reduction
| Metric | Before | After | Reduction |
|--------|--------|-------|-----------|
| **Allocation primitives** | 2 (`new`, `make`) | 1 (`&T{}`) | **50%** |
| **make() forms** | 3 (slice, map, chan) | 0 | **100%** |
| **Reference type special cases** | 3 types | 0 (unified) | **100%** |
| **Nil traps** | 2 (nil map write, nil chan) | 0 (consistent panic) | **100%** |
| **Type system categories** | 3 (value, ref, ptr) | 2 (value, ptr) | **33%** |
| **Syntax variants preserved** | Slicing, `<-`, select, range | All kept | **0%** |
**Total complexity reduction: ~30%** while keeping ergonomic syntax.
## Real-World Example: ORLY Codebase
### Before
```go
// pkg/database/query-events.go
func QueryEvents(db *badger.DB, filter *filter.T) ([]uint64, error) {
results := make([]uint64, 0, 1000)
// ... query logic
return results, nil
}
// Caller must handle returned slice
events, err := QueryEvents(db, f)
if err != nil {
return err
}
events = append(events, moreEvents...) // Might copy
```
### After
```go
// pkg/database/query-events.go
func QueryEvents(db *badger.DB, filter *filter.T) (results *[]uint64, err error) {
results = &[]uint64{}
results.Grow(1000) // Explicit capacity
// ... query logic
return
}
// Caller gets explicit pointer
events, err := QueryEvents(db, f)
if chk.E(err) {
return
}
events.Append(moreEvents...) // Clear mutation
```
**Benefits in ORLY:**
- Clear which functions mutate vs return new data
- Obvious when slices are shared across goroutines
- Explicit capacity management for performance-critical code
- No hidden allocations from append
## Conclusion
### What We Keep
✅ Slice expressions: `s[1:3:5]`
✅ Channel operators: `<-`
✅ Select statement
✅ For range channels
✅ Implicit append growth
✅ Convenient auto-dereference
### What We Gain
✅ Explicit pointer semantics
✅ Obvious data sharing
✅ Consistent nil behavior
✅ Unified type system
✅ Simpler language (no `make()`)
✅ Better concurrency safety
### What We Lose
`make()` function (replaced by `&T{}`)
❌ Implicit reference types (now explicit `*[]T`)
❌ Zero-value usability for maps/slices (must allocate)
### Recommendation
This revision strikes the right balance:
- **Keep** Go's ergonomic syntax that makes it productive
- **Add** explicit semantics that make code safer and clearer
- **Remove** only the truly confusing parts (`make()`, implicit references)
- **Gain** ~30% complexity reduction without sacrificing convenience
The migration is straightforward and could be done gradually with good tooling support.