This consolidates the pattern used for context overrides, notably
replacing clock overrides via experimental.WithTimeNowUnixNano
and making all context keys internal.
This also makes sure experimental example tests are handled the same
way, notably backfilling one for WithFS
This performs several changes to allow compilation config to be
centralized and scoped properly. The immediate effects are that we can
now process external types during `Runtime.CompileModule` instead of
doing so later during `Runtime.InstantiateModule`. Another nice side
effect is memory size problems can err at a source line instead of
having to be handled in several places.
There are some API effects to this, and to pay for them, some less used
APIs were removed. The "easy APIs" are left alone. For example, the APIs
to compile and instantiate a module from Go or Wasm in one step are left
alone.
Here are the changes, some of which are only for consistency. Rationale
is summarized in each point.
* ModuleBuilder.Build -> ModuleBuilder.Compile
* The result of this is similar to `CompileModule`, and pairs better
with `ModuleBuilder.Instantiate` which is like `InstantiateModule`.
* CompiledCode -> CompiledModule
* We punted on this name, the result is more than just code. This is
better I think and more consistent as it introduces less terms.
* Adds CompileConfig param to Runtime.CompileModule.
* This holds existing features and will have future ones, such as
mapping externtypes to uint64 for wasm that doesn't yet support it.
* Merges Runtime.InstantiateModuleWithConfig with Runtime.InstantiateModule
* This allows us to explain APIs in terms of implicit or explicit
compilation and config, vs implicit, kindof implicit, and explicit.
* Removes Runtime.InstantiateModuleFromCodeWithConfig
* Similar to above, this API only saves the compilation step and also
difficult to reason with from a name POV.
* RuntimeConfig.WithMemory(CapacityPages|LimitPages) -> CompileConfig.WithMemorySizer
* This allows all error handling to be attached to the source line
* This also allows someone to reduce unbounded memory while knowing
what its minimum is.
* ModuleConfig.With(Import|ImportModule) -> CompileConfig.WithImportRenamer
* This allows more types of import manipulation, also without
conflating functions with globals.
* Adds api.ExternType
* Needed for ImportRenamer and will be needed later for ExportRenamer.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This commit adds support for multiple tables per module.
Notably, if the WithFeatureReferenceTypes is enabled,
call_indirect, table.init and table.copy instructions
can reference non-zero indexed tables.
part of #484
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This makes wazero.ModuleConfig an interface instead of a struct to
prevent it from being used incorrectly. For example, even though the
fields are not exported, someone can mistakenly instantiate this
when it is a struct, and in doing so violate internal assumptions.
This also fixes a mutability bug in the implementation.
Follow-up from #519 and the last in this series
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This makes wazero.CompiledCode an interface instead of a struct to
prevent it from being used incorrectly. For example, even though the
fields are not exported, someone can mistakenly instantiate this
when it is a struct, and in doing so violate internal assumptions.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
WebAssembly Core Working Draft 1 recently came out. Before that, we had
a toe-hold feature bucked called FinishedFeatures. This replaces
`RuntimeConfig.WithFinishedFeatures` with `RuntimeConfig.WithWasmCore2`.
This also adds `WithWasmCore1` for those who want to lock into 1.0
features as opposed to relying on defaults.
This also fixes some design debt where we hadn't finished migrating
public types that require constructor functions (NewXxx) to interfaces.
By using interfaces, we prevent people from accidentally initializing
key configuration directly (via &Xxx), causing nil field refs. This also
helps prevent confusion about how to use the type (ex pointer or not) as
there's only one way (as an interface).
See https://github.com/tetratelabs/wazero/issues/516
Signed-off-by: Adrian Cole <adrian@tetrate.io>
`Runtime.WithMemoryCapacityPages` is a function that determines memory
capacity in pages (65536 bytes per page). The inputs are the min and
possibly nil max defined by the module, and the default is to return
the min.
Ex. To set capacity to max when exists:
```golang
c.WithMemoryCapacityPages(func(minPages uint32, maxPages *uint32) uint32 {
if maxPages != nil {
return *maxPages
}
return minPages
})
```
Note: This applies at compile time, ModuleBuilder.Build or Runtime.CompileModule.
Fixes#500
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This prepares for exposing operations like Memory.Grow while keeping the
ability to trace what did that, by adding a `context.Context` initial
parameter. This adds this to all API methods that mutate or return
mutated data.
Before, we made a change to trace functions and general lifecycle
commands, but we missed this part. Ex. We track functions, but can't
track what closed the module, changed memory or a mutable constant.
Changing to do this now is not only more consistent, but helps us
optimize at least the interpreter to help users identify otherwise
opaque code that can cause harm. This is critical before we add more
functions that can cause harm, such as Memory.Grow.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This is an API breaking change that does a few things:
* Stop encouraging practice that can break context propagation:
* Stops caching `context.Context` in `wazero.RuntimeConfig`
* Stops caching `context.Context` in `api.Module`
* Fixes context propagation in function calls:
* Changes `api.Function`'s arg0 from `api.Module` to `context.Context`
* Adds `context.Context` parameter in instantiation (propagates to
.start)
* Allows context propagation for heavy operations like compile:
* Adds `context.Context` as the initial parameter of `CompileModule`
The design we had earlier was a good start, but this is the only way to
ensure coherence when users start correlating or tracing. While adding a
`context.Context` parameter may seem difficult, wazero is a low-level
library and WebAssembly is notoriously difficult to troubleshoot. In
other words, it will be easier to explain to users to pass (even nil) as
the context parameter vs try to figure out things without coherent
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This starts the process of removing all dependencies from wazero, by
isolating all assertions we use into a single file. This allows us to
port those assertions as we have time, and when twitchy is gone, the
project literally has no dependencies except go!
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Before, complicated wasm could be hard to implement, particularly as it
might have cyclic imports. This change allows users to re-map imports to
untangle any cycles or to break up monolithic modules like "env".
Signed-off-by: Adrian Cole <adrian@tetrate.io>
During #425, @neilalexander gave constructive feedback that the API is
both moving fast, and not good enough yet. This attempts to reduce the
incidental complexity at the cost of a little conflation.
### odd presence of `wasm` and `wasi` packages -> `api` package
We had public API packages in wasm and wasi, which helped us avoid
leaking too many internals as public. That these had names that look
like there should be implementations in them cause unnecessary
confusion. This squashes both into one package "api" which has no
package collission with anything.
We've long struggled with the poorly specified and non-uniformly
implemented WASI specification. Trying to bring visibility to its
constraints knowing they are routinely invalid taints our API for no
good reason. This removes all `WASI` commands for a default to invoke
the function `_start` if it exists. In doing so, there's only one path
to start a module.
Moreover, this puts all wasi code in a top-level package "wasi" as it
isn't re-imported by any internal types.
### Reuse of Module for pre and post instantiation to `Binary` -> `Module`
Module is defined by WebAssembly in many phases, from decoded to
instantiated. However, using the same noun in multiple packages is very
confusing. We at one point tried a name "DecodedModule" or
"InstantiatedModule", but this is a fools errand. By deviating slightly
from the spec we can make it unambiguous what a module is.
This make a result of compilation a `Binary`, retaining `Module` for an
instantiated one. In doing so, there's no longer any name conflicts
whatsoever.
### Confusion about config -> `ModuleConfig`
Also caused by splitting wasm into wasm+wasi is configuration. This
conflates both into the same type `ModuleConfig` as it is simpler than
trying to explain a "will never be finished" api of wasi snapshot-01 in
routine use of WebAssembly. In other words, this further moves WASI out
of the foreground as it has been nothing but burden.
```diff
--- a/README.md
+++ b/README.md
@@ -49,8 +49,8 @@ For example, here's how you can allow WebAssembly modules to read
-wm, err := r.InstantiateModule(wazero.WASISnapshotPreview1())
-defer wm.Close()
+wm, err := wasi.InstantiateSnapshotPreview1(r)
+defer wm.Close()
-sysConfig := wazero.NewSysConfig().WithFS(os.DirFS("/work/home"))
-module, err := wazero.StartWASICommandWithConfig(r, compiled, sysConfig)
+config := wazero.ModuleConfig().WithFS(os.DirFS("/work/home"))
+module, err := r.InstantiateModule(binary, config)
defer module.Close()
...
```
This allows users to reduce the memory limit per module below 4 Gi. This
is often needed because Wasm routinely leaves off the max, which implies
spec max (4 Gi). This uses Ki Gi etc in error messages because the spec
chooses to, though we can change to make it less awkward.
This also fixes an issue where we instantiated an engine inside config.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This introduces `SysConfig` to replace `WASIConfig` and formalize documentation around system calls.
The only incompatible change planned after this is to switch from wasi.FS to fs.FS
Implementation Notes:
Defaulting to os.Stdin os.Stdout and os.Stderr doesn't make sense for
the same reasons as why we don't propagate ENV or ARGV: it violates
sand-boxing. Moreover, these are worse as they prevent concurrency and
can also lead to console overload if accidentally not overridden.
This also changes default stdin to read EOF as that is safer than reading
from os.DevNull, which can run the host out of file descriptors.
Finally, this removes "WithPreopens" for "WithFS" and "WithWorkDirFS",
to focus on the intended result. Similar Docker, if the WorkDir isn't set, it
defaults to the same as root.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Allows users to do the following to enable sign-extension-ops:
```
r := wazero.NewRuntimeWithConfig(wazero.NewRuntimeConfig().WithFeatureSignExtensionOps(true))
```
Resolves#66
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
This adds `RuntimeConfig.WithFeatureMutableGlobal(enabled bool)`, which
allows disabling of mutable globals. When disabled, any attempt to add a
mutable global, either explicitly or implicitly via decoding wasm will
fail.
To support this, there's a new `Features` bitflag that can allow up to
63 feature toggles without passing structs.
While here, I fixed a significant performance problem in decoding
binary:
Before
```
BenchmarkCodecExample/binary.DecodeModule-16 184243 5623 ns/op 3848 B/op 184 allocs/op
```
Now
```
BenchmarkCodecExample/binary.DecodeModule-16 294084 3520 ns/op 2176 B/op 91 allocs/op
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
