The PR introduces the `platform.Select()` API, wrapping `select(2)` on POSIX and emulated in some cases on Windows. RATIONALE.md contains a full explanation of the approach followed in `poll_oneoff` to handle Stdin and the other types of file descriptors, and the clock subscriptions.
It also introduces an abstraction (`StdioFilePoller`) to allow the simulation of different scenarios (waiting for input, input ready, timeout expired, etc.) when unit-testing interactive input.
This closes#1317.
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Ensure that stdio device modes are consistent with the given
file descriptors by stat'ing, instead of returning mocks.
* Use `Stat()` on `poll_oneoff()` too, instead of `IsTerminal()`,
thus avoiding a useless syscall.
* Delete leftover type decl `fileModeStat`.
* Remove IsPlatform()
* Propagate error when Stat() fails
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
This forces all syscall functions, notably filesystem, to return numeric
codes as opposed to mapping in two different areas. The result of this
change is better consolidation in call sites of `sysfs.FS`, while
further refactoring is needed to address consolidation of file errors.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
wasi-testsuite changed its mind on pre-open WebAssembly/wasi-testsuite#66
they also now explicitly forbid paths being passed in with a leading slash. Even when the config bug on WebAssembly/wasi-testsuite#67 is finished, this requires discussion if we want to EPERM on that.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This forces all syscall functions, notably filesystem, to return numeric
codes as opposed to mapping in two different areas. The result of this
change is better consolidation in call sites of `sysfs.FS`, while
further refactoring is needed to address consolidation of file errors.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This returns stat as a value instead of a pointer param. This is both
more efficient and faster. It is also more efficient than returning a
pointer to a stat.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
gojs: implements lstat
This implements platform.Lstat and uses it in GOOS=js. Notably,
directory listings need to run lstat on their entries to get the correct
inodes back. In GOOS=js, directories are a fan-out of names, then lstat.
This also fixes stat for inodes on directories. We were missing a test
so we didn't know it was broken on windows. The approach used now is
reliable on go 1.20, and we should suggest anyone using windows to
compile with go 1.20.
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
We formerly cached only the directory type, to avoid re-stat'ing the
same directory many times. Since we are there, we can also cache the
inode, which is strictly required by wasi and costly to fetch. Note:
this only affects the directory: its contents still need a potential
fan-out of stats which will be handled in another change.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
wasi_snapshot_preview1 recently requires fd_readdir to return actual
inode values. On zero, wasi-libc will call fdstat to retrieve them.
This introduces our own `platform.Dirent` type and `Readdir` function
which a later change will allow fetching of inodes.
See https://github.com/WebAssembly/wasi-libc/pull/345
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This centralizes filestat logic by making our own `Stat_t` similar to
`syscall.Stat_t`. This exposes utilities in the platform package and
adds a new function `FS.Stat` which avoids having to use `fs.File` to
get the same info. Doing so at the FS abstraction allows us to optimize
how it is implemented internally using portable means (e.g.
`os.StatFile`) or OS-specific means where necessary, e.g. in windows.
This also ensures `platform.OpenFile` returns syscall.Errno and
centralizes error checking with a new `require.EqualErrno` test.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements WASI `sched_yield` with `sys.Osyield` that defaults to
return immediately. This is intentionally left without a built-in
alternative as common platforms such as darwin implement
`runtime.osyield` by sleeping for a microsecond. If we implemented that,
user code would be slowed down without a clear reason why.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This moves our floor version to the same we'll release 1.0 with: 1.18.
This is congruent with our version policy which is current-2.
Fixes#921
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This allows rewinding a directory by specifying the cookie 0, after
already scrolling a directory. This allows zig and clang functionality
to work which expect this.
This also fixes the case where a directory was allowed to be seeked.
This is prohibited by the current version of wasi-testsuite.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This implements fd_filestat_set_size and fd_filestat_set_times, which
passes one more test in the rust wasi-testsuite.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
This allows wasi to see mounts as individual preopens instead of
virtualized under a root. (GOOS=js doesn't use pre-opens so it still
sees a virtual root). This allows us to progress wasi-testsuite rust
tests, but acknowledges a current glitch in zig support, which is
tracked separately as #1077
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds a new top-level type FSConfig, which is configured via
`ModuleConfig.WithFSConfig(fcfg)`. This implements read-only and
read-write directory mounts, something not formally supported before. It
also implements `WithFS` which adapts a normal `fs.FS`. For convenience,
we retain the old `ModuleConfig.WithFS` signature so as to not affect
existing users much. A new configuration for our emerging raw
filesystem, `FSConfig.WithSysfs()` will happen later without breaking
this API.
Here's an example:
```
moduleConfig = wazero.NewModuleConfig().
// Make the current directory read-only accessible to the guest.
WithReadOnlyDirMount(".", "/")
// Make "/tmp/wasm" accessible to the guest as "/tmp".
WithDirMount("/tmp/wasm", "/tmp")
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This decouples sysfs.FS from fs.FS by introducing a temporary type
FSHolder, which will be removed when we top-level FSConfig (shortly).
This further reduces complexity by consolidating guest path
configuration into the only type that uses it: CompositeFS.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
It will help for us to rename earlier vs later, and syscallfs will be
laborious, especially after we introduce an FSConfig type and need to
declare a method name that differentiates from normal fs.FS. e.g. WithFS
vs WithSysFS reads nicer than WithSyscallFS, and meanwhile sys is
already a public package.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This reduces some boilerplate by extracting UnimplementedFS from the
existing FS implementations, such that it returns ENOSYS. This also
removes inconsistency where some methods on FS returned syscall.Errno
and others PathError.
Note: this doesn't get rid of all PathError, yet. We still need to
create a syscallfs.File type which would be able to do that. This is
just one preliminary cleanup before refactoring out the `fs.FS`
embedding from `syscallfs.DS`.
P.S. naming convention is arbitrary, so I took UnimplementedXXX from
grpc. This pattern is used a lot of places, also proxy-wasm-go-sdk, e.g.
`DefaultVMContext`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This prepares for pseudo-root when the CLI doesn't provide one by
improving the error messages in general, as well being consistent about
parameter order.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This rewrites compositeFS to syscallfs.FS following wasi-sdk preopen
rules. Notably, this allows use of read-only mounts now.
For example,
```bash
$ GOOS=js GOARCH=wasm bin/go test -c -o template.wasm text/template
$ wazero run -mount=src/text/template:/ -mount=/tmp:/tmp template.wasm -test.v
=== RUN TestExecute
--- PASS: TestExecute (0.07s)
--snip--
```
This is the first step to native WASI handling of multiple pre-opens.
After this change, it is still the case that there's only one pre-open
FD visible to wasm. A later change will make it possible for WASI to see
multiple pre-opens while `GOOS=js` which doesn't use preopens, remains
on a rootFS.
A future PR may need to add a CLI flag to disable escaping directories,
(e.g. make ../.. EINVAL), similar to `fs.FS` in Go. The simplest way to
allow this is to use a host-side RootFS even in WASI, and wrap that with
a `syscallfs` filename filter.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds FS.Path which holds the pre-open path currently only used in
WASI. It also fixes a TODO where we didn't know for sure if the FD
parameter for `path_` functions must always be a pre-open. The TL;DR; is
that usually it is, but it may not be (e.g. in our zig-cc example we can
see any directory FD, not just pre-opens).
Finally, this fixes a bug in our path resolution where we mistook paths
like "foo/foo" for "foo" because we only considered basenames instead of
the full path from the pre-open root.
This also makes pre-open directory lookup lazy because I noticed in
Trivy specifically, this is unnecessary for us to do eagerly, as they
change the FS at runtime per-call. In other words, any value from init
time is invalid later.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This consolidates internal code to syscallfs, which removes the fs.FS
specific path rules, except when adapting one to syscallfs. For example,
this allows the underlying filesystem to decide if relative paths are
supported or not, as well any EINVAL related concerns.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This stubs all remaining syscalls for `GOARCH=wasm GOOS=js` to return
ENOSYS, instead of panic'ing. This allows us to see the parameters it
receives.
For example:
```
==> go.syscall/js.valueCall(fs.truncate(path=/tmp/_Go_TestTruncate135754730,length=0))
<== (err=function not implemented,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
As noted in slack, we are unlikley to long term use fs.FS internally.
This ensures we attempt to cast to syscallfs.FS for all I/O by panicing
on fs.Open.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This fixes and backfills tests for open flags, notably O_CREAT. I
verified this with TinyGo tests, which now proceed when attempting to
create new files.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This renames the internal writefs package to syscallfs as it is largely
dependent on syscall signatures. This also implements utimes in gojs.
WASI will be a follow-up change as it requires more infrastructure.
Notably, we also need non-TinyGo tests because TinyGo doesn't yet
support os.Chtimes or corresponding syscalls.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This implements path_(create|remove)_directory path_unlink_file in wasi, particularly needed to use TinyGo tests to verify our interpretation of WASI. Use of this requires the experimental `writefs.DirFS`.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This splits unlink and rmdir from remove, as it is not only more precise
in GOOS=js, but it is also needed to implement wasi. I verified this
works by running go unit tests with logging.
```
==> go.syscall/js.valueCall(fs.open(name=/tmp/TestErrIsNotExist1062486353,flags=,perm=----------))
<== (err=<nil>,fd=10)
==> go.syscall/js.valueCall(fs.fstat(fd=10))
<== (err=<nil>,stat={isDir=true,mode=-rwx------,size=96,mtimeMs=1672285985206})
==> go.syscall/js.valueCall(fs.readdir(name=/tmp/TestErrIsNotExist1062486353))
<== (err=<nil>,dirents=&{[001]})
==> go.syscall/js.valueCall(fs.unlink(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=is a directory,ok=true)
==> go.syscall/js.valueCall(fs.rmdir(path=/tmp/TestErrIsNotExist1062486353/001))
<== (err=<nil>,ok=false)
```
Signed-off-by: Adrian Cole <adrian@tetrate.io>
The type we use to expose write operations is still evolving. It might
be a single writefs.FS interface, or similar to go where we have an
interface per feature (e.g. writefs.MkdirFS). These choices are all
implementation details for DirFS and won't be settled before the end of
the month version cutoff. Instead, this only exposes the ability to
create a DirFS, not an arbitrary implementation of writefs.FS. This does
so by making `writefs.FS` an internal type.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This adds writefs.FS, allowing functions to create and delete files.
This begins by implementing them on `GOARCH=js GOOS=wasm`. The current
status is a lot farther than before, even if completing write on WASI is
left for a later PR (possibly by another volunteer).
Signed-off-by: Adrian Cole <adrian@tetrate.io>
Allows wasm to close stdio file descriptors
This allows wasm to close stdio file descriptors such as STDOUT(1).
This will not close the underlying host resource as that would break a
lot of folks doing logging to the console.
Fixes#953
Signed-off-by: Adrian Cole <adrian@tetrate.io>
This uses ioctl syscalls or appropriate alternative, to detect if
stdin/out/err are character devices or not. This caches the result, to
ensure performance is ok at runtime as executing stat can approach
microsecond overhead.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.
For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.
Signed-off-by: Adrian Cole <adrian@tetrate.io>
