This starts the process of removing all dependencies from wazero, by
isolating all assertions we use into a single file. This allows us to
port those assertions as we have time, and when twitchy is gone, the
project literally has no dependencies except go!
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit implements all the arm64 instruction encodings necessary
for our JIT compiler and replaces the golang-asm assembler with our
handmade assembler on arm64 platform. Notably, this allows us to do
concurrent compilations.
This closes#233 combined with #406.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
Global constants can be defined in wasm or in ModuleBuilder. In either
case, they end up being decoded and interpreted during instantiation.
This chooses signed encoding to avoid surprises. A more comprehensive
explanation was added to RATIONALE.md, but the motivation was a global
100 coming out negative.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adjusts towards the exiting code which used int32/64 instead of
uint32/64. The reason is that the spec indicates intepretation as signed
numbers, which affects the maximum value.
See https://www.w3.org/TR/wasm-core-1/#value-types%E2%91%A2
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds functions to configure memory with ModuleBuilder. This uses
two functions, ExportMemory and ExportMemoryWithMax, as working with
uint32 pointers is awkward.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves the responsibility to close a module from the engine to the
context. The reason for this is that the engine is what defines the
function. When a module is closed, it is often from an imported host
function. If it is on the module engine, it is easy to accidentally
close an imported module.
This refactors the WASI tests also, to ensure they aren't cheating too
much. This allows us to know for example "proc_exit" works without too
much trouble.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This adds tests that pass without changing deferred error handling.
There are some tests that don't pass even without deferred error
handling. I'll add those in a separate PR.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
During #425, @neilalexander gave constructive feedback that the API is
both moving fast, and not good enough yet. This attempts to reduce the
incidental complexity at the cost of a little conflation.
### odd presence of `wasm` and `wasi` packages -> `api` package
We had public API packages in wasm and wasi, which helped us avoid
leaking too many internals as public. That these had names that look
like there should be implementations in them cause unnecessary
confusion. This squashes both into one package "api" which has no
package collission with anything.
We've long struggled with the poorly specified and non-uniformly
implemented WASI specification. Trying to bring visibility to its
constraints knowing they are routinely invalid taints our API for no
good reason. This removes all `WASI` commands for a default to invoke
the function `_start` if it exists. In doing so, there's only one path
to start a module.
Moreover, this puts all wasi code in a top-level package "wasi" as it
isn't re-imported by any internal types.
### Reuse of Module for pre and post instantiation to `Binary` -> `Module`
Module is defined by WebAssembly in many phases, from decoded to
instantiated. However, using the same noun in multiple packages is very
confusing. We at one point tried a name "DecodedModule" or
"InstantiatedModule", but this is a fools errand. By deviating slightly
from the spec we can make it unambiguous what a module is.
This make a result of compilation a `Binary`, retaining `Module` for an
instantiated one. In doing so, there's no longer any name conflicts
whatsoever.
### Confusion about config -> `ModuleConfig`
Also caused by splitting wasm into wasm+wasi is configuration. This
conflates both into the same type `ModuleConfig` as it is simpler than
trying to explain a "will never be finished" api of wasi snapshot-01 in
routine use of WebAssembly. In other words, this further moves WASI out
of the foreground as it has been nothing but burden.
```diff
--- a/README.md
+++ b/README.md
@@ -49,8 +49,8 @@ For example, here's how you can allow WebAssembly modules to read
-wm, err := r.InstantiateModule(wazero.WASISnapshotPreview1())
-defer wm.Close()
+wm, err := wasi.InstantiateSnapshotPreview1(r)
+defer wm.Close()
-sysConfig := wazero.NewSysConfig().WithFS(os.DirFS("/work/home"))
-module, err := wazero.StartWASICommandWithConfig(r, compiled, sysConfig)
+config := wazero.ModuleConfig().WithFS(os.DirFS("/work/home"))
+module, err := r.InstantiateModule(binary, config)
defer module.Close()
...
```
This allows users to reduce the memory limit per module below 4 Gi. This
is often needed because Wasm routinely leaves off the max, which implies
spec max (4 Gi). This uses Ki Gi etc in error messages because the spec
chooses to, though we can change to make it less awkward.
This also fixes an issue where we instantiated an engine inside config.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This exposes `wasm.Module.Name()` so that it is more coherent to
redefine a module with the same name, or know the name to import.
This uses that to port existing tests to use the new hammer
infrastracture (without module name collision).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This extracts `hammer.Hammer` as a utility for re-use, notably adding a
feature that ensures all tests run concurrently. Before, tests start in
a loop that could be delayed due to goroutine sheduling.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors tests that hammer shared state in ways that use locks or
atomics, so that they are consistent and also follow practice internally
used by Go itself. Notably, this supports the `-test.short` flag also.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This changes the way the JIT engine is tested by focusing on two areas:
* Did NewModuleEngine set a finalizer for each compiled function
correctly?
* Did ModuleEngine.Close remove its compiled function correctly?
A later change will do parallel tests for both engines and won't have to
at the same time check how finalizers work.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This deduplicates Engine tests so that there's less chance of copy/paste
errors and less requirement to use ad-hoc tests which are outside the
relevant source tree.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This backfills tests that ModuleContext.Close ends up calling
`Store.CloseModule` and `SysContext.Close`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This introduces `SysConfig` to replace `WASIConfig` and formalize documentation around system calls.
The only incompatible change planned after this is to switch from wasi.FS to fs.FS
Implementation Notes:
Defaulting to os.Stdin os.Stdout and os.Stderr doesn't make sense for
the same reasons as why we don't propagate ENV or ARGV: it violates
sand-boxing. Moreover, these are worse as they prevent concurrency and
can also lead to console overload if accidentally not overridden.
This also changes default stdin to read EOF as that is safer than reading
from os.DevNull, which can run the host out of file descriptors.
Finally, this removes "WithPreopens" for "WithFS" and "WithWorkDirFS",
to focus on the intended result. Similar Docker, if the WorkDir isn't set, it
defaults to the same as root.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds the skeleton implementation of amd64 assembler.
The assembler is not able to produce binary at all, and the actual
encoding implementations follow in subsequent commits.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
The prior design had a problem, where multiple imports of WASI would end
up having different file descriptors for the same file. Moreover, there
was no means to close any of these when a module importing WASI was
closed.
This moves all existing functionality to a new type SystemContext, which
is owned by the importing module, similar to how it owns its memory.
While this PR doesn't fix some problems such as unclosed files, the code
is now organized in a way it can be, and these issues will be resolved
by #394.
In order to fix scope, `WASISnapshotPreview1WithConfig` had to be
removed.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit makes compiler implementations build tag free.
Notably, now the arm64 and amd64 compilers are compilable
on any architecture. Anything arch-specific is isolated in files
named with arch_* prefixes.
This reduces the pain to make some broad change across
multiple architectures and allow developers to notice errors
on whichever architecture they are working on.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This introduces Assembler interface to abstract away golang-asm
behind it. Notably, new packages asm, asm/amd64 and asm/arm64
are added under internal/jit. Now golang-asm is only used in three files
golang_asm.go, amd64/golang_asm.go and arm64/golang_asm.go
where we implement amd64.Assembler and arm64.Assembler interface.
part of #233
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This enforces all ElementSegment rules prior to creating a new module
engine. This allows flexibility and less errors inside that function.
This also uses interface as table element to pass tests with -race
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
Before, if there was an unexpected panic on munmap of a code section,
we'd only see "invalid argument" and have to guess why. This propagates
context and stubs out the finalizer so we can test a better error, like:
"jit: failed to munmap code segment for test.function[0]: invalid argument"
This also attempts to improve troubleshooting time by making all module
instantiation that used empty module name, use the test name instead.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
StartWASICommandWithConfig is like StartWASICommand, except you can override configuration based on the importing
module. For example, you can use this to define different args depending on the importing module.
```go
// Initialize base configuration:
r := wazero.NewRuntime()
config := wazero.NewWASIConfig().WithStdout(buf)
wasi, _ := r.NewHostModule(wazero.WASISnapshotPreview1WithConfig(config))
decoded, _ := r.CompileModule(source)
// Assign configuration only when ready to instantiate.
module, _ := StartWASICommandWithConfig(r, decoded, config.WithArgs("rotate", "angle=90", "dir=cw"))
```
This also changes the config to be immutable since it may be reused many
times. Instead of assigning fields directly, use `WithXXX`.
Finally, this changes the examples to encourage using `.Close()`
Fixes#364
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit adds Module.Close API, which can be used to indicate that
the module will no longer be used and setting finalizers for the allocated
resources. Notably, this will set finalizers on compiledFunction to deallocate
mapped memory regions as well as removing them from the store-wide
storage. As a result, users can re-instantiate a module for the same name
while having the safety -- Calling Module.Close is safe even when there are
outstanding function calls as we never explicitly deallocate but indirectly
do that via Goruntine's finalizers.
resolves: #293
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This flattens Memory and Table types, particularly making it a
compilation error to add multiple of either.
This also backfills binary encoding of Table.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit fixes a few bugs in engine.Release, and backfills
unittests. Notably, moduleEngines must not release the imported
functions.
Signed-off-by: Takeshi Yoneda takeshi@tetrate.io
This commit adds ModuleEngine interface which is used to
make calls and created per-module instance. Notably, this
enables us to remove the necessity for store to holds
FunctionInstances and FunctionIndex.
This is a follow-up from #342
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
