This deduplicates examples, leaving only the most maintained or targeted
ones. Notably, this makes each runnable, avoiding test dependencies.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit implements all the arm64 instruction encodings necessary
for our JIT compiler and replaces the golang-asm assembler with our
handmade assembler on arm64 platform. Notably, this allows us to do
concurrent compilations.
This closes#233 combined with #406.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
Before, complicated wasm could be hard to implement, particularly as it
might have cyclic imports. This change allows users to re-map imports to
untangle any cycles or to break up monolithic modules like "env".
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Global constants can be defined in wasm or in ModuleBuilder. In either
case, they end up being decoded and interpreted during instantiation.
This chooses signed encoding to avoid surprises. A more comprehensive
explanation was added to RATIONALE.md, but the motivation was a global
100 coming out negative.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adjusts towards the exiting code which used int32/64 instead of
uint32/64. The reason is that the spec indicates intepretation as signed
numbers, which affects the maximum value.
See https://www.w3.org/TR/wasm-core-1/#value-types%E2%91%A2
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds functions to configure memory with ModuleBuilder. This uses
two functions, ExportMemory and ExportMemoryWithMax, as working with
uint32 pointers is awkward.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves the responsibility to close a module from the engine to the
context. The reason for this is that the engine is what defines the
function. When a module is closed, it is often from an imported host
function. If it is on the module engine, it is easy to accidentally
close an imported module.
This refactors the WASI tests also, to ensure they aren't cheating too
much. This allows us to know for example "proc_exit" works without too
much trouble.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This adds tests that pass without changing deferred error handling.
There are some tests that don't pass even without deferred error
handling. I'll add those in a separate PR.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
During #425, @neilalexander gave constructive feedback that the API is
both moving fast, and not good enough yet. This attempts to reduce the
incidental complexity at the cost of a little conflation.
### odd presence of `wasm` and `wasi` packages -> `api` package
We had public API packages in wasm and wasi, which helped us avoid
leaking too many internals as public. That these had names that look
like there should be implementations in them cause unnecessary
confusion. This squashes both into one package "api" which has no
package collission with anything.
We've long struggled with the poorly specified and non-uniformly
implemented WASI specification. Trying to bring visibility to its
constraints knowing they are routinely invalid taints our API for no
good reason. This removes all `WASI` commands for a default to invoke
the function `_start` if it exists. In doing so, there's only one path
to start a module.
Moreover, this puts all wasi code in a top-level package "wasi" as it
isn't re-imported by any internal types.
### Reuse of Module for pre and post instantiation to `Binary` -> `Module`
Module is defined by WebAssembly in many phases, from decoded to
instantiated. However, using the same noun in multiple packages is very
confusing. We at one point tried a name "DecodedModule" or
"InstantiatedModule", but this is a fools errand. By deviating slightly
from the spec we can make it unambiguous what a module is.
This make a result of compilation a `Binary`, retaining `Module` for an
instantiated one. In doing so, there's no longer any name conflicts
whatsoever.
### Confusion about config -> `ModuleConfig`
Also caused by splitting wasm into wasm+wasi is configuration. This
conflates both into the same type `ModuleConfig` as it is simpler than
trying to explain a "will never be finished" api of wasi snapshot-01 in
routine use of WebAssembly. In other words, this further moves WASI out
of the foreground as it has been nothing but burden.
```diff
--- a/README.md
+++ b/README.md
@@ -49,8 +49,8 @@ For example, here's how you can allow WebAssembly modules to read
-wm, err := r.InstantiateModule(wazero.WASISnapshotPreview1())
-defer wm.Close()
+wm, err := wasi.InstantiateSnapshotPreview1(r)
+defer wm.Close()
-sysConfig := wazero.NewSysConfig().WithFS(os.DirFS("/work/home"))
-module, err := wazero.StartWASICommandWithConfig(r, compiled, sysConfig)
+config := wazero.ModuleConfig().WithFS(os.DirFS("/work/home"))
+module, err := r.InstantiateModule(binary, config)
defer module.Close()
...
```
This allows users to reduce the memory limit per module below 4 Gi. This
is often needed because Wasm routinely leaves off the max, which implies
spec max (4 Gi). This uses Ki Gi etc in error messages because the spec
chooses to, though we can change to make it less awkward.
This also fixes an issue where we instantiated an engine inside config.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This exposes `wasm.Module.Name()` so that it is more coherent to
redefine a module with the same name, or know the name to import.
This uses that to port existing tests to use the new hammer
infrastracture (without module name collision).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This extracts `hammer.Hammer` as a utility for re-use, notably adding a
feature that ensures all tests run concurrently. Before, tests start in
a loop that could be delayed due to goroutine sheduling.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors tests that hammer shared state in ways that use locks or
atomics, so that they are consistent and also follow practice internally
used by Go itself. Notably, this supports the `-test.short` flag also.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This changes the way the JIT engine is tested by focusing on two areas:
* Did NewModuleEngine set a finalizer for each compiled function
correctly?
* Did ModuleEngine.Close remove its compiled function correctly?
A later change will do parallel tests for both engines and won't have to
at the same time check how finalizers work.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This deduplicates Engine tests so that there's less chance of copy/paste
errors and less requirement to use ad-hoc tests which are outside the
relevant source tree.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Currently, we have custom code in wapc-go because our library forces a
failure when a module that uses WASI doesn't define a "_start" function.
Using the same pragmatism that resulted in us not enforcing the WASI
table, this makes the "_start" function optional. This doesn't add a
flag as the spec is not a proper version anyway (snapshot-01), so
there's no need to further complicate configuration.
If a "_start" function exists, we enforce it is of the proper signature
and succeeds. Otherwise, we allow it to be absent.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This backfills tests that ModuleContext.Close ends up calling
`Store.CloseModule` and `SysContext.Close`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This introduces `SysConfig` to replace `WASIConfig` and formalize documentation around system calls.
The only incompatible change planned after this is to switch from wasi.FS to fs.FS
Implementation Notes:
Defaulting to os.Stdin os.Stdout and os.Stderr doesn't make sense for
the same reasons as why we don't propagate ENV or ARGV: it violates
sand-boxing. Moreover, these are worse as they prevent concurrency and
can also lead to console overload if accidentally not overridden.
This also changes default stdin to read EOF as that is safer than reading
from os.DevNull, which can run the host out of file descriptors.
Finally, this removes "WithPreopens" for "WithFS" and "WithWorkDirFS",
to focus on the intended result. Similar Docker, if the WorkDir isn't set, it
defaults to the same as root.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds the skeleton implementation of amd64 assembler.
The assembler is not able to produce binary at all, and the actual
encoding implementations follow in subsequent commits.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
The prior design had a problem, where multiple imports of WASI would end
up having different file descriptors for the same file. Moreover, there
was no means to close any of these when a module importing WASI was
closed.
This moves all existing functionality to a new type SystemContext, which
is owned by the importing module, similar to how it owns its memory.
While this PR doesn't fix some problems such as unclosed files, the code
is now organized in a way it can be, and these issues will be resolved
by #394.
In order to fix scope, `WASISnapshotPreview1WithConfig` had to be
removed.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
go-wasm3 is the first implementation to beat wazero at a benchmark. Its
initialization time is less than half our time in our bench case.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves test-only overrides out of config in preparation of
normalized configuration. This also simplifies error tests by using the
API directly for all cases except exit.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit makes compiler implementations build tag free.
Notably, now the arm64 and amd64 compilers are compilable
on any architecture. Anything arch-specific is isolated in files
named with arch_* prefixes.
This reduces the pain to make some broad change across
multiple architectures and allow developers to notice errors
on whichever architecture they are working on.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
