gojs: implements lstat
This implements platform.Lstat and uses it in GOOS=js. Notably,
directory listings need to run lstat on their entries to get the correct
inodes back. In GOOS=js, directories are a fan-out of names, then lstat.
This also fixes stat for inodes on directories. We were missing a test
so we didn't know it was broken on windows. The approach used now is
reliable on go 1.20, and we should suggest anyone using windows to
compile with go 1.20.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This handles EOF even if current and possibly future wasi don't have a
way to propagate an EOF signal. This is mainly to match up with go
semantics and ensure we don't have any error conditions (by adding
tests).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds `platform.Readdirnames` which is preparation work before doing
something similar for reading the directory. We use this in gojs as it
doesn't actually need dirents, rather just names.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This centralizes filestat logic by making our own `Stat_t` similar to
`syscall.Stat_t`. This exposes utilities in the platform package and
adds a new function `FS.Stat` which avoids having to use `fs.File` to
get the same info. Doing so at the FS abstraction allows us to optimize
how it is implemented internally using portable means (e.g.
`os.StatFile`) or OS-specific means where necessary, e.g. in windows.
This also ensures `platform.OpenFile` returns syscall.Errno and
centralizes error checking with a new `require.EqualErrno` test.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds `FS.Chmod` and implements it for `GOOS=js`. This function
isn't defined in WASI snapshot01, but it is in `wasi-filesystem`, e.g.
`change-file-permissions-at`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements `fd_datasync` in WASI, falling back to normal
`File.Sync` when unsupported. This also backfills missing usage of sync
in GOOS=js. Finally, this updates the WASI status chart based on what's
implemented.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves our floor version to the same we'll release 1.0 with: 1.18.
This is congruent with our version policy which is current-2.
Fixes#921
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements fd_filestat_set_size and fd_filestat_set_times, which
passes one more test in the rust wasi-testsuite.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This allows you to specify the memory scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=memory,filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopeMemory|logging.LogScopeFilesystem))
```
This is helpful for emscripten and gojs which have memory reset
callbacks. This will be much more interesting once #1075 is implemented.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify the exit scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=exit,filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopeExit|logging.LogScopeFilesystem))
```
This is helpful to know if the wasm called exit or if it exited
implicitly. This is one of the few host functions that exists in three
places: assemblyscript, gojs and wasi.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify the poll scope amongst existing logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=poll --hostlogging=filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopePoll|logging.LogScopeFilesystem))
```
This is particularly helpful to understand side-effects of GC. For
example, in `GOOS=js` GC will trigger events and these have been tricky
to troubleshoot in the past.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This is to avoid a collision with an emerging wasi-crypto. They will
have both wasi-random and wasi-crypto
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows you to specify multiple logging scopes, both in API and the CLI.
e.g for the CLI.
```bash
$ wazero run --hostlogging=crypto --hostlogging=filesystem --mount=.:/:ro cat.wasm
```
e.g. for Go
```go
loggingCtx := context.WithValue(testCtx, experimental.FunctionListenerFactoryKey{},
logging.NewHostLoggingListenerFactory(&log, logging.LogScopeCrypto|logging.LogScopeFilesystem))
```
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Adrian Cole <adrian@tetrate.io>
This adds a new top-level type FSConfig, which is configured via
`ModuleConfig.WithFSConfig(fcfg)`. This implements read-only and
read-write directory mounts, something not formally supported before. It
also implements `WithFS` which adapts a normal `fs.FS`. For convenience,
we retain the old `ModuleConfig.WithFS` signature so as to not affect
existing users much. A new configuration for our emerging raw
filesystem, `FSConfig.WithSysfs()` will happen later without breaking
this API.
Here's an example:
```
moduleConfig = wazero.NewModuleConfig().
// Make the current directory read-only accessible to the guest.
WithReadOnlyDirMount(".", "/")
// Make "/tmp/wasm" accessible to the guest as "/tmp".
WithDirMount("/tmp/wasm", "/tmp")
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This decouples sysfs.FS from fs.FS by introducing a temporary type
FSHolder, which will be removed when we top-level FSConfig (shortly).
This further reduces complexity by consolidating guest path
configuration into the only type that uses it: CompositeFS.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
It will help for us to rename earlier vs later, and syscallfs will be
laborious, especially after we introduce an FSConfig type and need to
declare a method name that differentiates from normal fs.FS. e.g. WithFS
vs WithSysFS reads nicer than WithSyscallFS, and meanwhile sys is
already a public package.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This prepares for pseudo-root when the CLI doesn't provide one by
improving the error messages in general, as well being consistent about
parameter order.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements stat device and inode for WASI and GOOS=js, though it
does not implement the host side for windows, yet. Doing windows
requires plumbing as the values needed aren't exposed in Go. When we
re-do the syscallfs file type to have a stat method, we can address that
glitch. Meanwhile, I can find no Go sourcebase that does any better,
though the closest is the implementation details of os.SameFile.
I verified this with wasi-testsuite which now passes all but 1 case
which is unrelated (we haven't yet implemented `lseek`).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements WriterAtOffset, which supports WASI `fd_pwrite` and gojs
`fs.write` with offset, which is used to implement `syscall.Pwrite`.
I confirmed this passes the corresponding test in wasi-testsuite as
well.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This extracts a utility `syscallfs.ReaderAtOffset()` to allow WASI and
gojs to re-use the same logic to implement `syscall.Pread`.
What's different than before is that if WASI passes multiple iovecs an
emulated `ReaderAt` will seek to the read position on each call to
`Read` vs once per loop. This was a design decision to keep the call
sites compatible between files that implement ReaderAt and those that
emulate them with Seeker (e.g. avoid the need for a read-scoped closer/
defer function). The main use case for emulation is `embed.file`, whose
seek function is cheap, so there's little performance impact to this.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This rewrites compositeFS to syscallfs.FS following wasi-sdk preopen
rules. Notably, this allows use of read-only mounts now.
For example,
```bash
$ GOOS=js GOARCH=wasm bin/go test -c -o template.wasm text/template
$ wazero run -mount=src/text/template:/ -mount=/tmp:/tmp template.wasm -test.v
=== RUN TestExecute
--- PASS: TestExecute (0.07s)
--snip--
```
This is the first step to native WASI handling of multiple pre-opens.
After this change, it is still the case that there's only one pre-open
FD visible to wasm. A later change will make it possible for WASI to see
multiple pre-opens while `GOOS=js` which doesn't use preopens, remains
on a rootFS.
A future PR may need to add a CLI flag to disable escaping directories,
(e.g. make ../.. EINVAL), similar to `fs.FS` in Go. The simplest way to
allow this is to use a host-side RootFS even in WASI, and wrap that with
a `syscallfs` filename filter.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Formerly, we introduced `wazero.Namespace` to help avoid module name or import conflicts while still sharing the runtime's compilation cache. Now that we've introduced `CompilationCache` `wazero.Namespace` is no longer necessary. By removing it, we reduce the conceptual load on end users as well internal complexity. Since most users don't use namespace, the change isn't very impactful.
Users who are only trying to avoid module name conflict can generate a name like below instead of using multiple runtimes:
```go
moduleName := fmt.Sprintf("%d", atomic.AddUint64(&m.instanceCounter, 1))
module, err := runtime.InstantiateModule(ctx, compiled, config.WithName(moduleName))
```
For `HostModuleBuilder` users, we no longer take `Namespace` as the last parameter of `Instantiate` method:
```diff
// log to the console.
_, err := r.NewHostModuleBuilder("env").
NewFunctionBuilder().WithFunc(logString).Export("log").
- Instantiate(ctx, r)
+ Instantiate(ctx)
if err != nil {
log.Panicln(err)
}
```
The following is an example diff a use of namespace can use to keep compilation cache while also ensuring their modules don't conflict:
```diff
func useMultipleRuntimes(ctx context.Context, cache) {
- r := wazero.NewRuntime(ctx)
+ cache := wazero.NewCompilationCache()
for i := 0; i < N; i++ {
- // Create a new namespace to instantiate modules into.
- ns := r.NewNamespace(ctx) // Note: this is closed when the Runtime is
+ r := wazero.NewRuntimeWithConfig(ctx, wazero.NewRuntimeConfig().WithCompilationCache(cache))
// Instantiate a new "env" module which exports a stateful function.
_, err := r.NewHostModuleBuilder("env").
```
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
This introduces the new API wazero.Cache interface which can be passed to wazero.RuntimeConfig.
Users can configure this to share the underlying compilation cache across multiple wazero.Runtime.
And along the way, this deletes the experimental file cache API as it's replaced by this new API.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: Crypt Keeper <64215+codefromthecrypt@users.noreply.github.com>
This consolidates test files and ensures our various implementations of
`syscallfs.FS` pass `fstest.TestFS`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds FS.Path which holds the pre-open path currently only used in
WASI. It also fixes a TODO where we didn't know for sure if the FD
parameter for `path_` functions must always be a pre-open. The TL;DR; is
that usually it is, but it may not be (e.g. in our zig-cc example we can
see any directory FD, not just pre-opens).
Finally, this fixes a bug in our path resolution where we mistook paths
like "foo/foo" for "foo" because we only considered basenames instead of
the full path from the pre-open root.
This also makes pre-open directory lookup lazy because I noticed in
Trivy specifically, this is unnecessary for us to do eagerly, as they
change the FS at runtime per-call. In other words, any value from init
time is invalid later.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This avoids logging activity on stdio file descriptors, in order to help
make troubleshooting easier. Usually, there isn't an issue in these, yet
wasm panics are harder to read if there is also logging of the ..
logging.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates internal code to syscallfs, which removes the fs.FS
specific path rules, except when adapting one to syscallfs. For example,
this allows the underlying filesystem to decide if relative paths are
supported or not, as well any EINVAL related concerns.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This stubs all remaining syscalls for `GOARCH=wasm GOOS=js` to return
ENOSYS, instead of panic'ing. This allows us to see the parameters it
receives.
For example:
```
==> go.syscall/js.valueCall(fs.truncate(path=/tmp/_Go_TestTruncate135754730,length=0))
<== (err=function not implemented,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
As noted in slack, we are unlikley to long term use fs.FS internally.
This ensures we attempt to cast to syscallfs.FS for all I/O by panicing
on fs.Open.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This renames the internal writefs package to syscallfs as it is largely
dependent on syscall signatures. This also implements utimes in gojs.
WASI will be a follow-up change as it requires more infrastructure.
Notably, we also need non-TinyGo tests because TinyGo doesn't yet
support os.Chtimes or corresponding syscalls.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This splits unlink and rmdir from remove, as it is not only more precise
in GOOS=js, but it is also needed to implement wasi. I verified this
works by running go unit tests with logging.
```
==> go.syscall/js.valueCall(fs.open(name=/tmp/TestErrIsNotExist1062486353,flags=,perm=----------))
<== (err=<nil>,fd=10)
==> go.syscall/js.valueCall(fs.fstat(fd=10))
<== (err=<nil>,stat={isDir=true,mode=-rwx------,size=96,mtimeMs=1672285985206})
==> go.syscall/js.valueCall(fs.readdir(name=/tmp/TestErrIsNotExist1062486353))
<== (err=<nil>,dirents=&{[001]})
==> go.syscall/js.valueCall(fs.unlink(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=is a directory,ok=true)
==> go.syscall/js.valueCall(fs.rmdir(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=<nil>,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
The type we use to expose write operations is still evolving. It might
be a single writefs.FS interface, or similar to go where we have an
interface per feature (e.g. writefs.MkdirFS). These choices are all
implementation details for DirFS and won't be settled before the end of
the month version cutoff. Instead, this only exposes the ability to
create a DirFS, not an arbitrary implementation of writefs.FS. This does
so by making `writefs.FS` an internal type.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds writefs.FS, allowing functions to create and delete files.
This begins by implementing them on `GOARCH=js GOOS=wasm`. The current
status is a lot farther than before, even if completing write on WASI is
left for a later PR (possibly by another volunteer).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
`GOARCH=wasm GOOS=js` defines parameter names in go source, and they are
indirectly related to the wasm parameter "sp". This creates a pseudo
name section so that we can access the parameter names. The alternative
would be adding a hack to normal FunctionDefinition, only used for gojs.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This refactors GOOS and GOARCH specific code into their own packages.
This allows logging interceptors to be built without cyclic package
dependencies.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This separate host from guest instantiation in ways similar to other
imports such as emscripten. Doing so allows parallel use of gojs.Run,
provided the ModuleConfig has been assigned a unique name (e.g. via an
atomic number).
Fixes#939
Signed-off-by: Adrian Cole <adrian@tetrate.io>
