The PR introduces the `platform.Select()` API, wrapping `select(2)` on POSIX and emulated in some cases on Windows. RATIONALE.md contains a full explanation of the approach followed in `poll_oneoff` to handle Stdin and the other types of file descriptors, and the clock subscriptions.
It also introduces an abstraction (`StdioFilePoller`) to allow the simulation of different scenarios (waiting for input, input ready, timeout expired, etc.) when unit-testing interactive input.
This closes#1317.
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
This adds a wazero adapter which passes wasi-testsuite 100pct on darwin,
linux and windows. While the main change was adding inodes to the wasi
`fd_readdir` dirents, there was a lot of incidental work needed.
Most of the work was troubleshooting in nature, around windows
specifically, but also wrapping of files. This backfills a lot of tests
and reworked how wrapping works, particularly around windows.
To accommodate this, we drop `os.File` special casing except for
`sysfs.DirFS`
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements WASI `sched_yield` with `sys.Osyield` that defaults to
return immediately. This is intentionally left without a built-in
alternative as common platforms such as darwin implement
`runtime.osyield` by sleeping for a microsecond. If we implemented that,
user code would be slowed down without a clear reason why.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
The current fd_readdir test is invalid as it requires dot and dot-dot
entries, which both aren't required by POSIX nor returned by go. Once
this is addressed upstream, we can remove the skip.
See https://github.com/WebAssembly/wasi-testsuite/issues/52
See #1036
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This extracts a utility `syscallfs.ReaderAtOffset()` to allow WASI and
gojs to re-use the same logic to implement `syscall.Pread`.
What's different than before is that if WASI passes multiple iovecs an
emulated `ReaderAt` will seek to the read position on each call to
`Read` vs once per loop. This was a design decision to keep the call
sites compatible between files that implement ReaderAt and those that
emulate them with Seeker (e.g. avoid the need for a read-scoped closer/
defer function). The main use case for emulation is `embed.file`, whose
seek function is cheap, so there's little performance impact to this.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.
For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
At the moment, the only pre-open support we have is the file system
itself (root a.k.a. / or file-descriptor 3). We may in the future add
the ability to pre-open sockets, but in any case, this is where we are
today.
This change hardens logic around fd_preXXX functions, ensuring they only
work on actual pre-opens. This also fixes the path returned in filestat
as we sometimes returned a full path, when typically the basename is the
only part that can be returned.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We formerly introduced `MemorySizer` as a way to control capacity independently of size. This was the first and only feature in `CompileConfig`. While possibly used privately, `MemorySizer` has never been used in public GitHub code.
These APIs interfere with how we do caching of compiled modules. Notably, they can change the min or max defined in wasm, which invalidates some constants. This has also had a bad experience, forcing everyone to boilerplate`wazero.NewCompileConfig()` despite that API never being used in open source.
This addresses the use cases in a different way, by moving configuration to `RuntimeConfig` instead. This allows us to remove `MemorySizer` and `CompileConfig`, and the problems with them, yet still retaining functionality in case someone uses it.
* `RuntimeConfig.WithMemoryLimitPages(uint32)`: Prevents memory from growing to 4GB (spec limit) per instance.
* This works regardless of whether the wasm encodes max or not. If there is no max, it becomes effectively this value.
* `RuntimeConfig.WithMemoryCapacityFromMax(bool)`: Prevents reallocations (when growing).
* Wasm that never sets max will grow from min to the limit above.
Note: Those who want to change their wasm (ex insert a max where there was none), have to do that externally, ex via compiler settings or post-build transformations such as [wabin](https://github.com/tetratelabs/wabin)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We at one point considered making `ModuleBuilder` create complete
WebAssembly binaries. However, we recently spun out
[wabin](https://github.com/tetratelabs/wabin), which allows this.
Meanwhile, the features in `ModuleBuilder` were confusing and misused.
For example, the only two cases memory was exported on GitHub were done
by accident. This is because host functions act on the guest's memory,
not their own.
Hence, this removes memory and globals from host side definitions, and
renames the type to HostModuleBuilder to clarify this is not ever going
to be used to construct normal Wasm binaries.
Most importantly, this simplifies the API and reduces a lot of code. It
is important to make changes like this, particularly deleting any
experimental things that didn't end up useful.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Anuraag Agrawal <anuraaga@gmail.com>
While compilers should be conservative when targeting WebAssembly Core
features, runtimes should be lenient as otherwise people need to
constantly turn on all features. Currently, most examples have to turn
on 2.0 features because compilers such as AssemblyScript and TinyGo use
them by default. This matches the policy with the reality, and should
make first time use easier.
This top-levels an internal type as `api.CoreFeatures` and defaults to
2.0 as opposed to 1.0, our previous default. This is less cluttered than
the excess of `WithXXX` methods we had prior to implementing all
planned WebAssembly Core Specification 1.0 features.
Finally, this backfills rationale as flat config types were a distinct
decision even if feature set selection muddied the topic.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Before, we allowed stubbed host functions to be defined in wasm instead
of Go. This improves performance and reduces a chance of side-effects vs
Go. In fact, any pure function was supported in wasm, provided it only
called pure functions.
This changes internals so that a wasm-defined host function can use
memory. Notably, host functions use the caller's memory, so this is
simpler to initially support in the interpreter.
This is needed to simplify and reduce performance hit of GOARCH=wasm,
GOOS=js code, which perform a lot of memory reads and do not have
idiomatic signatures.
Note: wasm-defined host functions remain internal until we gain
experience, at least conclusion of the wasm_exec host module.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This changes the AssemblyScript abort handler and WASI proc_exit
implementation to panic the caller which eventually invoked close.
This ensures no code executes afterwards, For example, LLVM inserts
unreachable instructions after calls to exit.
See https://github.com/emscripten-core/emscripten/issues/12322
See #601
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes WithWorkDirFS and any other attempts to resolve the current directory (".") in host functions. This is a reaction to reality of compilers who track this inside wasm (not via host functions). One nice side effect is substantially simpler internal implementation of file-systems.
This also allows experimental.WithFS to block file access via passing nil.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements wasi_snapshot_preview1.poll_oneoff for relative clock events,
and in doing so stubs `Nanosleep` which defaults to noop, but can be configured
to `time.Sleep`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This ensures fake clocks increment so that compilers that implement
sleep with them don't spin.
This also fixes a mutability bug in config where we weren't really doing
clone properly because map references are shared.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds two clock interfaces: sys.Walltime and sys.Nanotime to
allow implementations to override readings for purposes of security or
determinism.
The default values of both are a fake timestamp, to avoid the sandbox
break we formerly had by returning the real time. This is similar to how
we don't inherit OS Env values.
This moves the platform-specific runtime code (currently only used by
the compiler) into its own package. Specifically, this moves the mmap
logic, and in doing so makes it easier to test, for example new
operating systems.
This also backfills missing RATIONALE about x/sys and hints at a future
possibility to allow a plugin. However, the next step is to get FreeBSD
working natively on the compiler without any additional dependencies.
See #607
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This notably changes NewRuntimeJIT to NewRuntimeCompiler as well renames
packages from jit to compiler.
This clarifies the implementation is AOT, not JIT, at least when
clarified to where it occurs (Runtime.CompileModule). In doing so, we
reduce any concern that compilation will happen during function
execution. We also free ourselves to create a JIT option without
confusion in the future via CompileConfig or otherwise.
Fixes#560
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Global constants can be defined in wasm or in ModuleBuilder. In either
case, they end up being decoded and interpreted during instantiation.
This chooses signed encoding to avoid surprises. A more comprehensive
explanation was added to RATIONALE.md, but the motivation was a global
100 coming out negative.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
During #425, @neilalexander gave constructive feedback that the API is
both moving fast, and not good enough yet. This attempts to reduce the
incidental complexity at the cost of a little conflation.
### odd presence of `wasm` and `wasi` packages -> `api` package
We had public API packages in wasm and wasi, which helped us avoid
leaking too many internals as public. That these had names that look
like there should be implementations in them cause unnecessary
confusion. This squashes both into one package "api" which has no
package collission with anything.
We've long struggled with the poorly specified and non-uniformly
implemented WASI specification. Trying to bring visibility to its
constraints knowing they are routinely invalid taints our API for no
good reason. This removes all `WASI` commands for a default to invoke
the function `_start` if it exists. In doing so, there's only one path
to start a module.
Moreover, this puts all wasi code in a top-level package "wasi" as it
isn't re-imported by any internal types.
### Reuse of Module for pre and post instantiation to `Binary` -> `Module`
Module is defined by WebAssembly in many phases, from decoded to
instantiated. However, using the same noun in multiple packages is very
confusing. We at one point tried a name "DecodedModule" or
"InstantiatedModule", but this is a fools errand. By deviating slightly
from the spec we can make it unambiguous what a module is.
This make a result of compilation a `Binary`, retaining `Module` for an
instantiated one. In doing so, there's no longer any name conflicts
whatsoever.
### Confusion about config -> `ModuleConfig`
Also caused by splitting wasm into wasm+wasi is configuration. This
conflates both into the same type `ModuleConfig` as it is simpler than
trying to explain a "will never be finished" api of wasi snapshot-01 in
routine use of WebAssembly. In other words, this further moves WASI out
of the foreground as it has been nothing but burden.
```diff
--- a/README.md
+++ b/README.md
@@ -49,8 +49,8 @@ For example, here's how you can allow WebAssembly modules to read
-wm, err := r.InstantiateModule(wazero.WASISnapshotPreview1())
-defer wm.Close()
+wm, err := wasi.InstantiateSnapshotPreview1(r)
+defer wm.Close()
-sysConfig := wazero.NewSysConfig().WithFS(os.DirFS("/work/home"))
-module, err := wazero.StartWASICommandWithConfig(r, compiled, sysConfig)
+config := wazero.ModuleConfig().WithFS(os.DirFS("/work/home"))
+module, err := r.InstantiateModule(binary, config)
defer module.Close()
...
```
This extracts `hammer.Hammer` as a utility for re-use, notably adding a
feature that ensures all tests run concurrently. Before, tests start in
a loop that could be delayed due to goroutine sheduling.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors tests that hammer shared state in ways that use locks or
atomics, so that they are consistent and also follow practice internally
used by Go itself. Notably, this supports the `-test.short` flag also.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
