Formerly, we introduced `wazero.Namespace` to help avoid module name or import conflicts while still sharing the runtime's compilation cache. Now that we've introduced `CompilationCache` `wazero.Namespace` is no longer necessary. By removing it, we reduce the conceptual load on end users as well internal complexity. Since most users don't use namespace, the change isn't very impactful.
Users who are only trying to avoid module name conflict can generate a name like below instead of using multiple runtimes:
```go
moduleName := fmt.Sprintf("%d", atomic.AddUint64(&m.instanceCounter, 1))
module, err := runtime.InstantiateModule(ctx, compiled, config.WithName(moduleName))
```
For `HostModuleBuilder` users, we no longer take `Namespace` as the last parameter of `Instantiate` method:
```diff
// log to the console.
_, err := r.NewHostModuleBuilder("env").
NewFunctionBuilder().WithFunc(logString).Export("log").
- Instantiate(ctx, r)
+ Instantiate(ctx)
if err != nil {
log.Panicln(err)
}
```
The following is an example diff a use of namespace can use to keep compilation cache while also ensuring their modules don't conflict:
```diff
func useMultipleRuntimes(ctx context.Context, cache) {
- r := wazero.NewRuntime(ctx)
+ cache := wazero.NewCompilationCache()
for i := 0; i < N; i++ {
- // Create a new namespace to instantiate modules into.
- ns := r.NewNamespace(ctx) // Note: this is closed when the Runtime is
+ r := wazero.NewRuntimeWithConfig(ctx, wazero.NewRuntimeConfig().WithCompilationCache(cache))
// Instantiate a new "env" module which exports a stateful function.
_, err := r.NewHostModuleBuilder("env").
```
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.
For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This switches to gofumpt and applies changes, as I've noticed working
in dapr (who uses this) that it finds some things that are annoying,
such as inconsistent block formatting in test tables.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This restores the ability to leave out the initial context parameter
when defining functions with reflection. This is important because some
projects are porting from a different library to wazero, and all the
alternatives are not contextualized.
For example, this project is porting envoy host functions, and the
original definitions (in mosn) don't have a context parameter. By being
lenient, they can migrate easier.
See 6b813482b6/pkg/proxywasm/wazero/imports_v1.go
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This PR follows @hafeidejiangyou advice to not only enable end users to
avoid reflection when calling host functions, but also use that approach
ourselves internally. The performance results are staggering and will be
noticable in high performance applications.
Before
```
BenchmarkHostCall/Call
BenchmarkHostCall/Call-16 1000000 1050 ns/op
Benchmark_EnvironGet/environGet
Benchmark_EnvironGet/environGet-16 525492 2224 ns/op
```
Now
```
BenchmarkHostCall/Call
BenchmarkHostCall/Call-16 14807203 83.22 ns/op
Benchmark_EnvironGet/environGet
Benchmark_EnvironGet/environGet-16 951690 1054 ns/op
```
To accomplish this, this PR consolidates code around host function
definition and enables a fast path for functions where the user takes
responsibility for defining its WebAssembly mappings. Existing users
will need to change their code a bit, as signatures have changed.
For example, we are now more strict that all host functions require a
context parameter zero. Also, we've replaced
`HostModuleBuilder.ExportFunction` and `ExportFunctions` with a new type
`HostFunctionBuilder` that consolidates the responsibility and the
documentation.
```diff
ctx := context.Background()
-hello := func() {
+hello := func(context.Context) {
fmt.Fprintln(stdout, "hello!")
}
-_, err := r.NewHostModuleBuilder("env").ExportFunction("hello", hello).Instantiate(ctx, r)
+_, err := r.NewHostModuleBuilder("env").
+ NewFunctionBuilder().WithFunc(hello).Export("hello").
+ Instantiate(ctx, r)
```
Power users can now use `HostFunctionBuilder` to define functions that
won't use reflection. There are two choices of interfaces to use
depending on if that function needs access to the calling module or not:
`api.GoFunction` and `api.GoModuleFunction`. Here's an example defining
one.
```go
builder.WithGoFunction(api.GoFunc(func(ctx context.Context, params []uint64) []uint64 {
x, y := uint32(params[0]), uint32(params[1])
sum := x + y
return []uint64{sum}
}, []api.ValueType{api.ValueTypeI32, api.ValueTypeI32}, []api.ValueType{api.ValueTypeI32})
```
As you'll notice and as documented, this approach is more verbose and
not for everyone. If you aren't making a low-level library, you are
likely able to afford the 1us penalty for the convenience of reflection.
However, we are happy to enable this option for foundational libraries
and those with high performance requirements (like ourselves)!
Fixes#825
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We formerly introduced `MemorySizer` as a way to control capacity independently of size. This was the first and only feature in `CompileConfig`. While possibly used privately, `MemorySizer` has never been used in public GitHub code.
These APIs interfere with how we do caching of compiled modules. Notably, they can change the min or max defined in wasm, which invalidates some constants. This has also had a bad experience, forcing everyone to boilerplate`wazero.NewCompileConfig()` despite that API never being used in open source.
This addresses the use cases in a different way, by moving configuration to `RuntimeConfig` instead. This allows us to remove `MemorySizer` and `CompileConfig`, and the problems with them, yet still retaining functionality in case someone uses it.
* `RuntimeConfig.WithMemoryLimitPages(uint32)`: Prevents memory from growing to 4GB (spec limit) per instance.
* This works regardless of whether the wasm encodes max or not. If there is no max, it becomes effectively this value.
* `RuntimeConfig.WithMemoryCapacityFromMax(bool)`: Prevents reallocations (when growing).
* Wasm that never sets max will grow from min to the limit above.
Note: Those who want to change their wasm (ex insert a max where there was none), have to do that externally, ex via compiler settings or post-build transformations such as [wabin](https://github.com/tetratelabs/wabin)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We at one point considered making `ModuleBuilder` create complete
WebAssembly binaries. However, we recently spun out
[wabin](https://github.com/tetratelabs/wabin), which allows this.
Meanwhile, the features in `ModuleBuilder` were confusing and misused.
For example, the only two cases memory was exported on GitHub were done
by accident. This is because host functions act on the guest's memory,
not their own.
Hence, this removes memory and globals from host side definitions, and
renames the type to HostModuleBuilder to clarify this is not ever going
to be used to construct normal Wasm binaries.
Most importantly, this simplifies the API and reduces a lot of code. It
is important to make changes like this, particularly deleting any
experimental things that didn't end up useful.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com>
wazero has no runtime deps. This updates tools we use as well as
dependencies used in comparison with other runtimes.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
While compilers should be conservative when targeting WebAssembly Core
features, runtimes should be lenient as otherwise people need to
constantly turn on all features. Currently, most examples have to turn
on 2.0 features because compilers such as AssemblyScript and TinyGo use
them by default. This matches the policy with the reality, and should
make first time use easier.
This top-levels an internal type as `api.CoreFeatures` and defaults to
2.0 as opposed to 1.0, our previous default. This is less cluttered than
the excess of `WithXXX` methods we had prior to implementing all
planned WebAssembly Core Specification 1.0 features.
Finally, this backfills rationale as flat config types were a distinct
decision even if feature set selection muddied the topic.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Rather than clutter both the README and home page with details that can
drift and have caused maintenance, this directs users to the tested
basic example.
This also adds a FAQ entry about TinyGo's main, thanks to @basvanbeek
for the help.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Our root directory is getting crowded and it is also difficult to
organize the "host imports" concept due to this.
This moves common and language-specific imports into their own
directory. This will break go import signatures on the next release, but
is more sustainable overall.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This updates to AssemblyScript 0.21 which no longer includes WASI. This
updates guidance to use the shim, which I verified works.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds an experimental package gojs which implements the host side of Wasm compiled by GOARCH=wasm GOOS=js go build -o X.wasm X.go
This includes heavy disclaimers, in part inherited by Go's comments https://github.com/golang/go/blob/go1.19/src/syscall/js/js.go#L10-L11
Due to this many will still use TinyGo instead.
That said, this is frequently asked for and has interesting features including reflection and HTTP client support.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates common notes we've had into a landing page.
This is intentionally only simplified content for TinyGo to make the
first review easier (as well the writing easier). After this, PRs will
happen to consolidate the other notes we have sporadically in the
examples directory.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
staticcheck linters broke until recent golangci-lint. Now, normal
behaviour of enforcing no nil context works again. Ex.
```
assemblyscript/assemblyscript_example_test.go:16:25: SA1012: do not pass a nil Context, even if a function permits it; pass context.TODO if you are unsure about which Context to use (staticcheck)
r := wazero.NewRuntime(nil)
```
Since default lint already checks for nil context, this removes our
permission of nil context args. The original reason we permitted nil is
no longer valid: we once allowed context to be stashed in config, and
removed that as it caused bugs. We forgot to undo allowing nil
explicitly.
Note: this doesn't particularly check in our code for nil context,
similar as we don't particularly check in our code for nil anything
else. End users should use linters as none of our parameters should be
nil anyway.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds a toehold integration for emscripten users, so they don't
have to define the memory growth function.
Fixes#601
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Before, we introduced a type `api.ImportRenamer` to resolve conflicts
where the "env" module was shared between AssemblyScript and
user-defined functions. This API was never used in GitHub, and is
complicated.
AssemblyScript also isn't the only ABI to share the "env" module, as
other web APIs like Emscripten do also. The less complicated approach is
to have packages that need to share "env" use
`ModuleBuilder.ExportFunctions` instead, and use namespaces as needed if
there is overlap.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This fixes where our pull requests didn't check the rust source in
examples were valid. It also adds an example of wasi with rust.
This uses cargo-wasi because the default target creates almost 2MB of
wasm for a simple cat program.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This changes the AssemblyScript abort handler and WASI proc_exit
implementation to panic the caller which eventually invoked close.
This ensures no code executes afterwards, For example, LLVM inserts
unreachable instructions after calls to exit.
See https://github.com/emscripten-core/emscripten/issues/12322
See #601
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes WithWorkDirFS and any other attempts to resolve the current directory (".") in host functions. This is a reaction to reality of compilers who track this inside wasm (not via host functions). One nice side effect is substantially simpler internal implementation of file-systems.
This also allows experimental.WithFS to block file access via passing nil.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds two clock interfaces: sys.Walltime and sys.Nanotime to
allow implementations to override readings for purposes of security or
determinism.
The default values of both are a fake timestamp, to avoid the sandbox
break we formerly had by returning the real time. This is similar to how
we don't inherit OS Env values.
This drops the text format (%.wat) and renames
InstantiateModuleFromCode to InstantiateModuleFromBinary as it is no
longer ambiguous.
We decided to stop supporting the text format as it isn't typically used
in production, yet costs a lot of work to develop. Given the resources
available and the increased work added with WebAssembly 2.0 and soon
WASI 2, we can't afford to spend the time on it.
The old parser is used only internally and will eventually be moved to
its own repository named watzero, possibly towards archival.
See #59
Signed-off-by: Adrian Cole <adrian@tetrate.io>
The componentized successor to wasi_snapshot_preview1 is not compatible
with the prior imports or even error numbers. Before releasing wazero
1.0 we need to change this package to reflect that WASI 2 is effectively
a different API.
Fixes#263
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates the pattern used for context overrides, notably
replacing clock overrides via experimental.WithTimeNowUnixNano
and making all context keys internal.
This also makes sure experimental example tests are handled the same
way, notably backfilling one for WithFS
This removes tedium in our examples and docs by using `Runtime.Close`
instead of tracking everything. Internal tests still track too much, but
anyway at least this stops suggesting others should do it.
This also changes our examples to use log.PanicXX so that the line
number goes into the console output.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
