This changes file descriptors from uint32 to int32 and the
corresponding file table to reject negative values. This ensures
invalid values aren't mistaken for very large descriptor entries, which
can use a lot of memory as the table implementation isn't designed to
be sparse.
See https://pubs.opengroup.org/onlinepubs/9699919799/functions/dirfd.html#tag_16_90
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify the memory scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=memory,filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopeMemory|logging.LogScopeFilesystem))
```
This is helpful for emscripten and gojs which have memory reset
callbacks. This will be much more interesting once #1075 is implemented.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify the exit scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=exit,filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopeExit|logging.LogScopeFilesystem))
```
This is helpful to know if the wasm called exit or if it exited
implicitly. This is one of the few host functions that exists in three
places: assemblyscript, gojs and wasi.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify the poll scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=poll --hostlogging=filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopePoll|logging.LogScopeFilesystem))
```
This is particularly helpful to understand side-effects of GC. For
example, in `GOOS=js` GC will trigger events and these have been tricky
to troubleshoot in the past.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates internal code to syscallfs, which removes the fs.FS
specific path rules, except when adapting one to syscallfs. For example,
this allows the underlying filesystem to decide if relative paths are
supported or not, as well any EINVAL related concerns.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
`GOARCH=wasm GOOS=js` defines parameter names in go source, and they are
indirectly related to the wasm parameter "sp". This creates a pseudo
name section so that we can access the parameter names. The alternative
would be adding a hack to normal FunctionDefinition, only used for gojs.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors GOOS and GOARCH specific code into their own packages.
This allows logging interceptors to be built without cyclic package
dependencies.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes ProxyFunc, which was an internal experiment to make
functions that use memory to store parameters or results easier to see.
The main issue with the approach was instantiation performance, as it
needs to dynamically generate functions. Another approach to visibility
can happen later, for example via internal logging hooks.
Notably, this fixed the performance regression after switching WASI to ProxyFunc:
```
name old time/op new time/op delta
Allocation/Compile-16 39.8ms ± 0% 39.5ms ± 0% -0.62% (p=0.016 n=4+5)
Allocation/Instantiate-16 1.74ms ± 4% 0.86ms ± 1% -50.19% (p=0.008 n=5+5)
Allocation/Call-16 4.25µs ± 1% 4.21µs ± 2% ~ (p=0.151 n=5+5)
name old alloc/op new alloc/op delta
Allocation/Compile-16 20.3MB ± 0% 20.3MB ± 0% ~ (p=0.841 n=5+5)
Allocation/Instantiate-16 1.04MB ± 0% 0.56MB ± 0% -45.88% (p=0.008 n=5+5)
Allocation/Call-16 48.0B ± 0% 48.0B ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
Allocation/Compile-16 432k ± 0% 432k ± 0% ~ (p=0.833 n=5+5)
Allocation/Instantiate-16 16.6k ± 0% 6.7k ± 0% -59.34% (p=0.008 n=5+5)
Allocation/Call-16 5.00 ± 0% 5.00 ± 0% ~ (all equal)
```
Since we also removed it from `GOARCH=wasm GOOS=js`, we experienced a performance
benefit there as well:
```
name old time/op new time/op delta
_main/gojs.Run-16 13.7ms ± 1% 12.2ms ± 3% -10.76% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
_main/gojs.Run-16 25.4MB ± 0% 25.0MB ± 0% -1.66% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
_main/gojs.Run-16 166k ± 0% 158k ± 0% -4.79% (p=0.016 n=4+5)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.
For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes the ability to override the current file system with Go
context, allowing us to simplify common paths and improve performance.
The context override was only used once in GitHub, in Trivy, and we
found another way to do that without it.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds ResultNames to HostFunctionBuilder and FunctionDefinition
which helps for multi-results or special-cased ones.
End users can access result names in `FunctionDefinition.ResultNames` or
set for their own host functions via
`HostFunctionBuilder.WithResultNames`. This change adds them for all
built-in functions where result names help.
Most notably, GOOS=js uses `ProxyFunc` to allow logging when a function
returns multiple results. Before, the results were returned without
names: e.g. `11231,1` and now they are named like `n=11231,ok=1`.
We soon plan to allow more visibility in WASI, for example, logging
results that will write to memory offsets. This infrastructure makes it
possible to do that.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This PR follows @hafeidejiangyou advice to not only enable end users to
avoid reflection when calling host functions, but also use that approach
ourselves internally. The performance results are staggering and will be
noticable in high performance applications.
Before
```
BenchmarkHostCall/Call
BenchmarkHostCall/Call-16 1000000 1050 ns/op
Benchmark_EnvironGet/environGet
Benchmark_EnvironGet/environGet-16 525492 2224 ns/op
```
Now
```
BenchmarkHostCall/Call
BenchmarkHostCall/Call-16 14807203 83.22 ns/op
Benchmark_EnvironGet/environGet
Benchmark_EnvironGet/environGet-16 951690 1054 ns/op
```
To accomplish this, this PR consolidates code around host function
definition and enables a fast path for functions where the user takes
responsibility for defining its WebAssembly mappings. Existing users
will need to change their code a bit, as signatures have changed.
For example, we are now more strict that all host functions require a
context parameter zero. Also, we've replaced
`HostModuleBuilder.ExportFunction` and `ExportFunctions` with a new type
`HostFunctionBuilder` that consolidates the responsibility and the
documentation.
```diff
ctx := context.Background()
-hello := func() {
+hello := func(context.Context) {
fmt.Fprintln(stdout, "hello!")
}
-_, err := r.NewHostModuleBuilder("env").ExportFunction("hello", hello).Instantiate(ctx, r)
+_, err := r.NewHostModuleBuilder("env").
+ NewFunctionBuilder().WithFunc(hello).Export("hello").
+ Instantiate(ctx, r)
```
Power users can now use `HostFunctionBuilder` to define functions that
won't use reflection. There are two choices of interfaces to use
depending on if that function needs access to the calling module or not:
`api.GoFunction` and `api.GoModuleFunction`. Here's an example defining
one.
```go
builder.WithGoFunction(api.GoFunc(func(ctx context.Context, params []uint64) []uint64 {
x, y := uint32(params[0]), uint32(params[1])
sum := x + y
return []uint64{sum}
}, []api.ValueType{api.ValueTypeI32, api.ValueTypeI32}, []api.ValueType{api.ValueTypeI32})
```
As you'll notice and as documented, this approach is more verbose and
not for everyone. If you aren't making a low-level library, you are
likely able to afford the 1us penalty for the convenience of reflection.
However, we are happy to enable this option for foundational libraries
and those with high performance requirements (like ourselves)!
Fixes#825
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds an experimental package gojs which implements the host side of Wasm compiled by GOARCH=wasm GOOS=js go build -o X.wasm X.go
This includes heavy disclaimers, in part inherited by Go's comments https://github.com/golang/go/blob/go1.19/src/syscall/js/js.go#L10-L11
Due to this many will still use TinyGo instead.
That said, this is frequently asked for and has interesting features including reflection and HTTP client support.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
