This notably changes NewRuntimeJIT to NewRuntimeCompiler as well renames
packages from jit to compiler.
This clarifies the implementation is AOT, not JIT, at least when
clarified to where it occurs (Runtime.CompileModule). In doing so, we
reduce any concern that compilation will happen during function
execution. We also free ourselves to create a JIT option without
confusion in the future via CompileConfig or otherwise.
Fixes#560
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit implements the v128.const, i32x4.add and i64x2.add in
interpreter mode and this adds support for the vector value types in the
locals and globals.
Notably, the vector type values can be passed and returned by exported functions
as well as host functions via two-uint64 encodings as described in #484 (comment).
Note: implementation of these instructions on JIT will be done in subsequent PR.
part of #484
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This commit enables WebAssembly 2.0 Core Specification tests.
In order to pass the tests, this fixes several places mostly on the
validation logic.
Note that SIMD instructions are not implemented yet.
part of #484
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Crypt Keeper <64215+codefromthecrypt@users.noreply.github.com>
This commit completes the reference-types proposal implementation.
Notably, this adds support for
* `ref.is_null`, `ref.func`, `ref.is_null` instructions
* `table.get`, `table.set`, `table.grow`, `table.size` and `table.fill` instructions
* `Externref` and `Funcref` types (including invocation via uint64 encoding).
part of #484
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
By testing the side-effects of Memory.Read, we ensure users who control
the underlying memory capacity can use the returned slice for
write-through access to Wasm addressible memory. Notably, this allows a
shared fixed length data structure to exist with a pointer on the Go
side and a memory offset on the Wasm side.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This commit adds support for multiple tables per module.
Notably, if the WithFeatureReferenceTypes is enabled,
call_indirect, table.init and table.copy instructions
can reference non-zero indexed tables.
part of #484
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This adds an experimental package to expose two work-in-progress
features:
* FunctionListener - for tracing etc.
* Sys - to control random number generators
Both the functionality and the names of the features above are
not stable. However, this should help those who can tolerate drift a
means to test things out.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com>
This commit implements the rest of the unimplemented instructions in the
bulk-memory-operations proposal.
Notably, this adds support for table.init, table.copy and elem.drop
instructions toggled by FeatureBulkMemoryOperations.
Given that, now wazero has the complete support for the bulk-memory-operations
proposal as described in https://github.com/WebAssembly/spec/blob/main/proposals/bulk-memory-operations/Overview.mdfixes#321
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This is an API breaking change that does a few things:
* Stop encouraging practice that can break context propagation:
* Stops caching `context.Context` in `wazero.RuntimeConfig`
* Stops caching `context.Context` in `api.Module`
* Fixes context propagation in function calls:
* Changes `api.Function`'s arg0 from `api.Module` to `context.Context`
* Adds `context.Context` parameter in instantiation (propagates to
.start)
* Allows context propagation for heavy operations like compile:
* Adds `context.Context` as the initial parameter of `CompileModule`
The design we had earlier was a good start, but this is the only way to
ensure coherence when users start correlating or tracing. While adding a
`context.Context` parameter may seem difficult, wazero is a low-level
library and WebAssembly is notoriously difficult to troubleshoot. In
other words, it will be easier to explain to users to pass (even nil) as
the context parameter vs try to figure out things without coherent
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This centralizes logic that gets params and results in and out of
Go-defined functions. This allows us to refactor and optimize easier as
well ensure value coersion is the same regardless of the engine type.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This commit allows CompiledCode to be re-used regardless of
the existence of import replacement configs for instantiation.
In order to achieve this, we introduce ModuleID, which is sha256
checksum calculated on source bytes, as a key for module compilation
cache. Previously, we used*wasm.Module as keys for caches which
differ before/after import replacement.
Signed-off-by: Takeshi Yoneda takeshi@tetrate.io
This commit makes it possible for functions to be compiled before instantiation.
Notably, this adds CompileModule method on Engine interface where we pass
wasm.Module (which is the decoded module) to engines, and engines compile
all the module functions and caches them keyed on *wasm.Module.
In order to achieve that, this stops the compiled native code from embedding typeID
which is assigned for all the function types in a store.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Before, we tested the examples/ directory using "ExampleXX", but this is
not ideal because it literally embeds the call to `main` into example
godoc output. This stops doing that for a different infrastructure.
This also makes sure there's a godoc example for both the main package
and wasi, so that people looking at https://pkg.go.dev see something and
also a link to our real examples directory.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements our own assertion library so that we can remove the
testify dependency. I've changed a lot of call sites so that we only
have to maintain a minimal amount of assertions.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Thanks to #454, now the compiled binary (code segment) can be reused for
multiple module instances originating from the same source (wasm.Module).
This commit introduces the caching mechanism on engine where it caches
compiled functions keyed on `wasm.Module`. As a result, this allows us to
do the fast module instantiation from the same *CompiledCode.
In order to release the cache properly, this also adds `Close` method
on CompiledCode.
Here's some bench result for instantiating multiple modules from the same CompiledCode:
```
name old time/op new time/op delta
Initialization/interpreter-32 2.84ms ± 3% 0.06ms ± 1% -97.73% (p=0.008 n=5+5)
Initialization/jit-32 10.7ms ±18% 0.1ms ± 1% -99.52% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Initialization/interpreter-32 1.25MB ± 0% 0.15MB ± 0% -88.41% (p=0.008 n=5+5)
Initialization/jit-32 4.46MB ± 0% 0.15MB ± 0% -96.69% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Initialization/interpreter-32 35.2k ± 0% 0.3k ± 0% -99.29% (p=0.008 n=5+5)
Initialization/jit-32 94.1k ± 0% 0.2k ± 0% -99.74% (p=0.008 n=5+5)
```
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This starts the process of removing all dependencies from wazero, by
isolating all assertions we use into a single file. This allows us to
port those assertions as we have time, and when twitchy is gone, the
project literally has no dependencies except go!
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves the responsibility to close a module from the engine to the
context. The reason for this is that the engine is what defines the
function. When a module is closed, it is often from an imported host
function. If it is on the module engine, it is easy to accidentally
close an imported module.
This refactors the WASI tests also, to ensure they aren't cheating too
much. This allows us to know for example "proc_exit" works without too
much trouble.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This adds tests that pass without changing deferred error handling.
There are some tests that don't pass even without deferred error
handling. I'll add those in a separate PR.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
During #425, @neilalexander gave constructive feedback that the API is
both moving fast, and not good enough yet. This attempts to reduce the
incidental complexity at the cost of a little conflation.
### odd presence of `wasm` and `wasi` packages -> `api` package
We had public API packages in wasm and wasi, which helped us avoid
leaking too many internals as public. That these had names that look
like there should be implementations in them cause unnecessary
confusion. This squashes both into one package "api" which has no
package collission with anything.
We've long struggled with the poorly specified and non-uniformly
implemented WASI specification. Trying to bring visibility to its
constraints knowing they are routinely invalid taints our API for no
good reason. This removes all `WASI` commands for a default to invoke
the function `_start` if it exists. In doing so, there's only one path
to start a module.
Moreover, this puts all wasi code in a top-level package "wasi" as it
isn't re-imported by any internal types.
### Reuse of Module for pre and post instantiation to `Binary` -> `Module`
Module is defined by WebAssembly in many phases, from decoded to
instantiated. However, using the same noun in multiple packages is very
confusing. We at one point tried a name "DecodedModule" or
"InstantiatedModule", but this is a fools errand. By deviating slightly
from the spec we can make it unambiguous what a module is.
This make a result of compilation a `Binary`, retaining `Module` for an
instantiated one. In doing so, there's no longer any name conflicts
whatsoever.
### Confusion about config -> `ModuleConfig`
Also caused by splitting wasm into wasm+wasi is configuration. This
conflates both into the same type `ModuleConfig` as it is simpler than
trying to explain a "will never be finished" api of wasi snapshot-01 in
routine use of WebAssembly. In other words, this further moves WASI out
of the foreground as it has been nothing but burden.
```diff
--- a/README.md
+++ b/README.md
@@ -49,8 +49,8 @@ For example, here's how you can allow WebAssembly modules to read
-wm, err := r.InstantiateModule(wazero.WASISnapshotPreview1())
-defer wm.Close()
+wm, err := wasi.InstantiateSnapshotPreview1(r)
+defer wm.Close()
-sysConfig := wazero.NewSysConfig().WithFS(os.DirFS("/work/home"))
-module, err := wazero.StartWASICommandWithConfig(r, compiled, sysConfig)
+config := wazero.ModuleConfig().WithFS(os.DirFS("/work/home"))
+module, err := r.InstantiateModule(binary, config)
defer module.Close()
...
```
This extracts `hammer.Hammer` as a utility for re-use, notably adding a
feature that ensures all tests run concurrently. Before, tests start in
a loop that could be delayed due to goroutine sheduling.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This changes the way the JIT engine is tested by focusing on two areas:
* Did NewModuleEngine set a finalizer for each compiled function
correctly?
* Did ModuleEngine.Close remove its compiled function correctly?
A later change will do parallel tests for both engines and won't have to
at the same time check how finalizers work.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This deduplicates Engine tests so that there's less chance of copy/paste
errors and less requirement to use ad-hoc tests which are outside the
relevant source tree.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
