This adds a new top-level type FSConfig, which is configured via
`ModuleConfig.WithFSConfig(fcfg)`. This implements read-only and
read-write directory mounts, something not formally supported before. It
also implements `WithFS` which adapts a normal `fs.FS`. For convenience,
we retain the old `ModuleConfig.WithFS` signature so as to not affect
existing users much. A new configuration for our emerging raw
filesystem, `FSConfig.WithSysfs()` will happen later without breaking
this API.
Here's an example:
```
moduleConfig = wazero.NewModuleConfig().
// Make the current directory read-only accessible to the guest.
WithReadOnlyDirMount(".", "/")
// Make "/tmp/wasm" accessible to the guest as "/tmp".
WithDirMount("/tmp/wasm", "/tmp")
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This decouples sysfs.FS from fs.FS by introducing a temporary type
FSHolder, which will be removed when we top-level FSConfig (shortly).
This further reduces complexity by consolidating guest path
configuration into the only type that uses it: CompositeFS.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
It will help for us to rename earlier vs later, and syscallfs will be
laborious, especially after we introduce an FSConfig type and need to
declare a method name that differentiates from normal fs.FS. e.g. WithFS
vs WithSysFS reads nicer than WithSyscallFS, and meanwhile sys is
already a public package.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This reduces some boilerplate by extracting UnimplementedFS from the
existing FS implementations, such that it returns ENOSYS. This also
removes inconsistency where some methods on FS returned syscall.Errno
and others PathError.
Note: this doesn't get rid of all PathError, yet. We still need to
create a syscallfs.File type which would be able to do that. This is
just one preliminary cleanup before refactoring out the `fs.FS`
embedding from `syscallfs.DS`.
P.S. naming convention is arbitrary, so I took UnimplementedXXX from
grpc. This pattern is used a lot of places, also proxy-wasm-go-sdk, e.g.
`DefaultVMContext`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This prepares for pseudo-root when the CLI doesn't provide one by
improving the error messages in general, as well being consistent about
parameter order.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This rewrites compositeFS to syscallfs.FS following wasi-sdk preopen
rules. Notably, this allows use of read-only mounts now.
For example,
```bash
$ GOOS=js GOARCH=wasm bin/go test -c -o template.wasm text/template
$ wazero run -mount=src/text/template:/ -mount=/tmp:/tmp template.wasm -test.v
=== RUN TestExecute
--- PASS: TestExecute (0.07s)
--snip--
```
This is the first step to native WASI handling of multiple pre-opens.
After this change, it is still the case that there's only one pre-open
FD visible to wasm. A later change will make it possible for WASI to see
multiple pre-opens while `GOOS=js` which doesn't use preopens, remains
on a rootFS.
A future PR may need to add a CLI flag to disable escaping directories,
(e.g. make ../.. EINVAL), similar to `fs.FS` in Go. The simplest way to
allow this is to use a host-side RootFS even in WASI, and wrap that with
a `syscallfs` filename filter.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds FS.Path which holds the pre-open path currently only used in
WASI. It also fixes a TODO where we didn't know for sure if the FD
parameter for `path_` functions must always be a pre-open. The TL;DR; is
that usually it is, but it may not be (e.g. in our zig-cc example we can
see any directory FD, not just pre-opens).
Finally, this fixes a bug in our path resolution where we mistook paths
like "foo/foo" for "foo" because we only considered basenames instead of
the full path from the pre-open root.
This also makes pre-open directory lookup lazy because I noticed in
Trivy specifically, this is unnecessary for us to do eagerly, as they
change the FS at runtime per-call. In other words, any value from init
time is invalid later.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates internal code to syscallfs, which removes the fs.FS
specific path rules, except when adapting one to syscallfs. For example,
this allows the underlying filesystem to decide if relative paths are
supported or not, as well any EINVAL related concerns.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This stubs all remaining syscalls for `GOARCH=wasm GOOS=js` to return
ENOSYS, instead of panic'ing. This allows us to see the parameters it
receives.
For example:
```
==> go.syscall/js.valueCall(fs.truncate(path=/tmp/_Go_TestTruncate135754730,length=0))
<== (err=function not implemented,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
As noted in slack, we are unlikley to long term use fs.FS internally.
This ensures we attempt to cast to syscallfs.FS for all I/O by panicing
on fs.Open.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This fixes and backfills tests for open flags, notably O_CREAT. I
verified this with TinyGo tests, which now proceed when attempting to
create new files.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This renames the internal writefs package to syscallfs as it is largely
dependent on syscall signatures. This also implements utimes in gojs.
WASI will be a follow-up change as it requires more infrastructure.
Notably, we also need non-TinyGo tests because TinyGo doesn't yet
support os.Chtimes or corresponding syscalls.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements path_(create|remove)_directory path_unlink_file in wasi, particularly needed to use TinyGo tests to verify our interpretation of WASI. Use of this requires the experimental `writefs.DirFS`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This splits unlink and rmdir from remove, as it is not only more precise
in GOOS=js, but it is also needed to implement wasi. I verified this
works by running go unit tests with logging.
```
==> go.syscall/js.valueCall(fs.open(name=/tmp/TestErrIsNotExist1062486353,flags=,perm=----------))
<== (err=<nil>,fd=10)
==> go.syscall/js.valueCall(fs.fstat(fd=10))
<== (err=<nil>,stat={isDir=true,mode=-rwx------,size=96,mtimeMs=1672285985206})
==> go.syscall/js.valueCall(fs.readdir(name=/tmp/TestErrIsNotExist1062486353))
<== (err=<nil>,dirents=&{[001]})
==> go.syscall/js.valueCall(fs.unlink(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=is a directory,ok=true)
==> go.syscall/js.valueCall(fs.rmdir(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=<nil>,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
The type we use to expose write operations is still evolving. It might
be a single writefs.FS interface, or similar to go where we have an
interface per feature (e.g. writefs.MkdirFS). These choices are all
implementation details for DirFS and won't be settled before the end of
the month version cutoff. Instead, this only exposes the ability to
create a DirFS, not an arbitrary implementation of writefs.FS. This does
so by making `writefs.FS` an internal type.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds writefs.FS, allowing functions to create and delete files.
This begins by implementing them on `GOARCH=js GOOS=wasm`. The current
status is a lot farther than before, even if completing write on WASI is
left for a later PR (possibly by another volunteer).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Allows wasm to close stdio file descriptors
This allows wasm to close stdio file descriptors such as STDOUT(1).
This will not close the underlying host resource as that would break a
lot of folks doing logging to the console.
Fixes#953
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This uses ioctl syscalls or appropriate alternative, to detect if
stdin/out/err are character devices or not. This caches the result, to
ensure performance is ok at runtime as executing stat can approach
microsecond overhead.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.
For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes the ability to override the current file system with Go
context, allowing us to simplify common paths and improve performance.
The context override was only used once in GitHub, in Trivy, and we
found another way to do that without it.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
At the moment, the only pre-open support we have is the file system
itself (root a.k.a. / or file-descriptor 3). We may in the future add
the ability to pre-open sockets, but in any case, this is where we are
today.
This change hardens logic around fd_preXXX functions, ensuring they only
work on actual pre-opens. This also fixes the path returned in filestat
as we sometimes returned a full path, when typically the basename is the
only part that can be returned.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
`os.DirFS` ironically doesn't implement `fs.ReadDirFS`, so we cannot
open a directory this way. This blindly attempts to open "." regardless
of if the `fs.FS` implementation is a `fs.ReadDirFS` or not, and if
successful, enforces that the file returned is a directory. If not, a
fake directory is returned.
Doing so allows real stat to be returned for root, and also a chance to
know if a filesystem configured is real or not. Later, we'll need this
to implement open flags.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Before, we didn't allow a real stat on the root file descriptor. Now,
those that pass fs.ReadDirFS will return the stat of the root file
(which is implemented by a open against ".").
This also simplifies logic as we always have a file representing root,
even if faked.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
While most compilers will only read args/environ once, tools like WAGI
make heavy use of environment, possibly dozens of long variables. This
optimizes both args and environ for this reason and also to setup for
optimizing other functions.
Here are the notable changes:
* eagerly coerce to byte slices instead of strings
* re-use null terminated length for writing values
* avoid loops that call mem.WriteXXX internally
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds an implementation of `fd_readdir` for WASI, which ensures a
very large directory is not kept in host memory until its directory is
closed.
Original implementation and test data are with thanks from @jerbob92.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: jerbob92 <jerbob92@users.noreply.github.com>
This switches to gofumpt and applies changes, as I've noticed working
in dapr (who uses this) that it finds some things that are annoying,
such as inconsistent block formatting in test tables.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This restores the ability to leave out the initial context parameter
when defining functions with reflection. This is important because some
projects are porting from a different library to wazero, and all the
alternatives are not contextualized.
For example, this project is porting envoy host functions, and the
original definitions (in mosn) don't have a context parameter. By being
lenient, they can migrate easier.
See 6b813482b6/pkg/proxywasm/wazero/imports_v1.go
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds an experimental package gojs which implements the host side of Wasm compiled by GOARCH=wasm GOOS=js go build -o X.wasm X.go
This includes heavy disclaimers, in part inherited by Go's comments https://github.com/golang/go/blob/go1.19/src/syscall/js/js.go#L10-L11
Due to this many will still use TinyGo instead.
That said, this is frequently asked for and has interesting features including reflection and HTTP client support.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This changes the default random source to provide deterministic values
similar to how nanotime and walltime do. This also prevents any worries
about if wasm can deplete the host's underlying source of entropy.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors host functions with no-op or constant returns to be
implemented with wasm instead of the host function bridge. This allows
better performance.
This also breaks up and makes WASI tests consistent, in a way that shows
parameter name drifts easier.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates to use EBADF in places go uses it in syscalls to
reduce where we formally returned both bool and err. This also removes
the redundant panic type handling as go will already panic with a
similar message.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This removes WithWorkDirFS and any other attempts to resolve the current directory (".") in host functions. This is a reaction to reality of compilers who track this inside wasm (not via host functions). One nice side effect is substantially simpler internal implementation of file-systems.
This also allows experimental.WithFS to block file access via passing nil.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements wasi_snapshot_preview1.poll_oneoff for relative clock events,
and in doing so stubs `Nanosleep` which defaults to noop, but can be configured
to `time.Sleep`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This ensures fake clocks increment so that compilers that implement
sleep with them don't spin.
This also fixes a mutability bug in config where we weren't really doing
clone properly because map references are shared.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds two clock interfaces: sys.Walltime and sys.Nanotime to
allow implementations to override readings for purposes of security or
determinism.
The default values of both are a fake timestamp, to avoid the sandbox
break we formerly had by returning the real time. This is similar to how
we don't inherit OS Env values.
The componentized successor to wasi_snapshot_preview1 is not compatible
with the prior imports or even error numbers. Before releasing wazero
1.0 we need to change this package to reflect that WASI 2 is effectively
a different API.
Fixes#263
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates the pattern used for context overrides, notably
replacing clock overrides via experimental.WithTimeNowUnixNano
and making all context keys internal.
This also makes sure experimental example tests are handled the same
way, notably backfilling one for WithFS
