mleku/wazero
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Removes GOOS=js related leftovers (#2193)

Browse Source 
Signed-off-by: Takeshi Yoneda <t.y.mathetake@gmail.com>
This commit is contained in:
Takeshi Yoneda
2024-04-22 11:17:10 +09:00
committed by GitHub
parent 5a9cb5aaf8
commit 7ad2b70626
21 changed files with 62 additions and 691 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
RATIONALE.md
View File

@@ -568,8 +568,7 @@ In short, wazero defined system configuration in `ModuleConfig`, not a WASI type
one spec to another with minimal impact. This has other helpful benefits, as centralized resources are simpler to close
coherently (ex via `Module.Close`).
In reflection, this worked well as more ABI became usable in wazero. For example, `GOOS=js GOARCH=wasm` code uses the
same `ModuleConfig` (and `FSConfig`) WASI uses, and in compatible ways.
In reflection, this worked well as more ABI became usable in wazero.
### Background on `ModuleConfig` design
@@ -753,8 +752,7 @@ independent of the root file system. This intended to help separate concerns
like mutability of files, but it didn't work and was removed.
Compilers that target wasm act differently with regard to the working
directory. For example, while `GOOS=js` uses host functions to track the
working directory, WASI host functions do not. wasi-libc, used by TinyGo,
directory. For example, wasi-libc, used by TinyGo,
tracks working directory changes in compiled wasm instead: initially "/" until
code calls `chdir`. Zig assumes the first pre-opened file descriptor is the
working directory.
@@ -829,10 +827,7 @@ The main reason is that `os.DirFS` is a virtual filesystem abstraction while
WASI is an abstraction over syscalls. For example, the signature of `fs.Open`
does not permit use of flags. This creates conflict on what default behaviors
to take when Go implemented `os.DirFS`. On the other hand, `path_open` can pass
flags, and in fact tests require them to be honored in specific ways. This
extends beyond WASI as even `GOOS=js GOARCH=wasm` compiled code requires
certain flags passed to `os.OpenFile` which are impossible to pass due to the
signature of `fs.FS`.
flags, and in fact tests require them to be honored in specific ways.
This conflict requires us to choose what to be more compatible with, and which
type of user to surprise the least. We assume there will be more developers
@@ -840,10 +835,6 @@ compiling code to wasm than developers of custom filesystem plugins, and those
compiling code to wasm will be better served if we are compatible with WASI.
Hence on conflict, we prefer WASI behavior vs the behavior of `os.DirFS`.
Meanwhile, it is possible that Go will one day compile to `GOOS=wasi` in
addition to `GOOS=js`. When there is shared stake in WASI, we expect gaps like
these to be easier to close.
See https://github.com/WebAssembly/wasi-testsuite
See https://github.com/golang/go/issues/58141
@@ -1205,10 +1196,7 @@ See https://gruss.cc/files/fantastictimers.pdf for an example attacks.
### Why does fake time increase on reading?
Both the fake nanotime and walltime increase by 1ms on reading. Particularly in
the case of nanotime, this prevents spinning. For example, when Go compiles
`time.Sleep` using `GOOS=js GOARCH=wasm`, nanotime is used in a loop. If that
never increases, the gouroutine is mistaken for being busy. This would be worse
if a compiler implement sleep using nanotime, yet doesn't check for spinning!
the case of nanotime, this prevents spinning.
### Why not `time.Clock`?
@@ -1266,8 +1254,7 @@ pub fn main() !void {
```
Besides Zig, this is also the case with TinyGo (`-target=wasi`) and Rust
(`--target wasm32-wasi`). This isn't the case with Go (`GOOS=js GOARCH=wasm`),
though. In the latter case, wasm loops on `sys.Nanotime`.
(`--target wasm32-wasi`).
We decided to expose `sys.Nanosleep` to allow overriding the implementation
used in the common case, even if it isn't used by Go, because this gives an

16
cmd/wazero/Dockerfile
View File

@@ -1,16 +0,0 @@
# This is an example Dockerfile used to show a way to integrate wasm.
FROM golang:1.21-alpine AS build
RUN apk add --no-cache git
WORKDIR /build
# wazero doesn't publish a Docker image or binary, yet, so build on on demand.
RUN git clone --depth 1 https://github.com/tetratelabs/wazero.git \
&& (cd wazero; go build -o wazero -ldflags "-w -s" ./cmd/wazero)
# wazero has no dependencies, so it can run on scratch
FROM scratch
COPY --from=build /build/wazero/wazero /bin/wazero
ENTRYPOINT ["/bin/wazero", "run", "-env-inherit", "-cachedir=/cache", "-mount=.:/"]

13
cmd/wazero/README.md
View File

@@ -20,16 +20,3 @@ wazero run calc.wasm 1 + 2
In addition to arguments, the WebAssembly binary has access to stdout, stderr,
and stdin.
### Docker / Podman
wazero doesn't currently publish binaries, but you can make your own with our
example [Dockerfile](Dockerfile). It should amount to about 4.5MB total.
```bash
# build the image
$ docker build -t wazero:latest -f Dockerfile .
# volume mount wasi or GOOS=js wasm you are interested in, and run it.
$ docker run -v $PWD/testdata/:/wasm wazero:latest /wasm/wasi_arg.wasm 1 2 3
wasi_arg.wasm123
```

2
cmd/wazero/testdata/cat/.gitignore vendored
View File

@@ -1,2 +1,2 @@
# GOOS=js GOARCH=wasm binaries are too huge to check-in
# GOOS=wasip1 GOARCH=wasm binaries are too huge to check-in
cat-go.wasm

35
config_test.go
View File

@@ -8,12 +8,9 @@ import (
"time"
"github.com/tetratelabs/wazero/api"
experimentalsys "github.com/tetratelabs/wazero/experimental/sys"
"github.com/tetratelabs/wazero/internal/fstest"
"github.com/tetratelabs/wazero/internal/platform"
internalsys "github.com/tetratelabs/wazero/internal/sys"
"github.com/tetratelabs/wazero/internal/sysfs"
testfs "github.com/tetratelabs/wazero/internal/testing/fs"
"github.com/tetratelabs/wazero/internal/testing/require"
"github.com/tetratelabs/wazero/internal/wasm"
"github.com/tetratelabs/wazero/sys"
@@ -319,38 +316,6 @@ func TestModuleConfig_toSysContext(t *testing.T) {
}
},
},
{
name: "WithFS",
input: func() (ModuleConfig, func(t *testing.T, sys *internalsys.Context)) {
testFS := &testfs.FS{}
config := base.WithFS(testFS)
return config, func(t *testing.T, sys *internalsys.Context) {
rootfs := sys.FS().RootFS()
require.Equal(t, &sysfs.AdaptFS{FS: testFS}, rootfs)
}
},
},
{
name: "WithFS overwrites",
input: func() (ModuleConfig, func(t *testing.T, sys *internalsys.Context)) {
testFS, testFS2 := &testfs.FS{}, &testfs.FS{}
config := base.WithFS(testFS).WithFS(testFS2)
return config, func(t *testing.T, sys *internalsys.Context) {
rootfs := sys.FS().RootFS()
require.Equal(t, &sysfs.AdaptFS{FS: testFS2}, rootfs)
}
},
},
{
name: "WithFS nil",
input: func() (ModuleConfig, func(t *testing.T, sys *internalsys.Context)) {
config := base.WithFS(nil)
return config, func(t *testing.T, sys *internalsys.Context) {
rootfs := sys.FS().RootFS()
require.Equal(t, experimentalsys.UnimplementedFS{}, rootfs)
}
},
},
{
name: "WithRandSource",
input: func() (ModuleConfig, func(t *testing.T, sys *internalsys.Context)) {

5
experimental/sys/file.go
View File

@@ -3,7 +3,7 @@ package sys
import "github.com/tetratelabs/wazero/sys"
// File is a writeable fs.File bridge backed by syscall functions needed for ABI
// including WASI and runtime.GOOS=js.
// including WASI.
//
// Implementations should embed UnimplementedFile for forward compatibility. Any
// unsupported method or parameter should return ENOSYS.
@@ -14,8 +14,7 @@ import "github.com/tetratelabs/wazero/sys"
// on success.
//
// Restricting to Errno matches current WebAssembly host functions,
// which are constrained to well-known error codes. For example, `GOOS=js` maps
// hard coded values and panics otherwise. More commonly, WASI maps syscall
// which are constrained to well-known error codes. For example, WASI maps syscall
// errors to u32 numeric values.
//
// # Notes

5
experimental/sys/fs.go
View File

@@ -7,7 +7,7 @@ import (
)
// FS is a writeable fs.FS bridge backed by syscall functions needed for ABI
// including WASI and runtime.GOOS=js.
// including WASI.
//
// Implementations should embed UnimplementedFS for forward compatibility. Any
// unsupported method or parameter should return ENO
@@ -18,8 +18,7 @@ import (
// on success.
//
// Restricting to Errno matches current WebAssembly host functions,
// which are constrained to well-known error codes. For example, `GOOS=js` maps
// hard coded values and panics otherwise. More commonly, WASI maps syscall
// which are constrained to well-known error codes. For example, WASI maps syscall
// errors to u32 numeric values.
//
// # Notes

2
experimental/sysfs/sysfs.go
View File

@@ -1,5 +1,5 @@
// Package sysfs includes a low-level filesystem interface and utilities needed
// for WebAssembly host functions (ABI) such as WASI and runtime.GOOS=js.
// for WebAssembly host functions (ABI) such as WASI.
//
// The name sysfs was chosen because wazero's public API has a "sys" package,
// which was named after https://github.com/golang/sys.

2
fsconfig.go
View File

@@ -30,8 +30,6 @@ import (
// current directory prior to requesting files.
//
// More notes on `guestPath`
// - Go compiled with runtime.GOOS=js do not pay attention to this value.
// It only works with root mounts ("").
// - Working directories are typically tracked in wasm, though possible some
// relative paths are requested. For example, TinyGo may attempt to resolve
// a path "../.." in unit tests.

4
imports/README.md
View File

@@ -33,7 +33,3 @@ Packages in this directory are sometimes well re-used, such as the case in
all target WebAssembly in a way that imports the same "wasi_snapshot_preview1"
module in the compiled `%.wasm` file. To support any of these, wazero users can
invoke `wasi_snapshot_preview1.Instantiate` on their `wazero.Runtime`.
Other times, host imports are either completely compiler-specific, such as the
case with `GOOS=js GOARCH=wasm`, or coexist alongside WASI, such as the case
with Emscripten.

47
imports/wasi_snapshot_preview1/fs_test.go
View File

@@ -54,7 +54,7 @@ func Test_fdAllocate(t *testing.T) {
wazero.NewFSConfig().WithDirMount(tmpDir, "/"),
))
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
defer r.Close(testCtx)
fd, errno := fsc.OpenFile(preopen, fileName, experimentalsys.O_RDWR, 0)
@@ -125,6 +125,12 @@ func Test_fdAllocate(t *testing.T) {
`, "\n"+log.String())
}
func getPreopen(t *testing.T, fsc *sys.FSContext) experimentalsys.FS {
preopen, ok := fsc.LookupFile(sys.FdPreopen)
require.True(t, ok)
return preopen.FS
}
func Test_fdClose(t *testing.T) {
// fd_close needs to close an open file descriptor. Open two files so that we can tell which is closed.
path1, path2 := "dir/-", "dir/a-"
@@ -133,7 +139,7 @@ func Test_fdClose(t *testing.T) {
// open both paths without using WASI
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fdToClose, errno := fsc.OpenFile(preopen, path1, experimentalsys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
@@ -237,7 +243,7 @@ func Test_fdFdstatGet(t *testing.T) {
// open both paths without using WASI
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
// replace stdin with a fake TTY file.
// TODO: Make this easier once we have in-memory sys.File
@@ -488,7 +494,7 @@ func Test_fdFdstatSetFlagsWithTrunc(t *testing.T) {
defer r.Close(testCtx)
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fd, errno := fsc.OpenFile(preopen, fileName, experimentalsys.O_RDWR|experimentalsys.O_CREAT|experimentalsys.O_EXCL|experimentalsys.O_TRUNC, 0o600)
require.EqualErrno(t, 0, errno)
@@ -533,7 +539,7 @@ func Test_fdFdstatSetFlags(t *testing.T) {
WithStderr(stderrW).
WithFSConfig(wazero.NewFSConfig().WithDirMount(tmpDir, "/")))
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
defer r.Close(testCtx)
// First, O_CREAT the file with O_APPEND. We use O_EXCL because that
@@ -663,7 +669,7 @@ func Test_fdFilestatGet(t *testing.T) {
// open both paths without using WASI
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fileFD, errno := fsc.OpenFile(preopen, file, experimentalsys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
@@ -2075,7 +2081,7 @@ func Test_fdReaddir(t *testing.T) {
defer r.Close(testCtx)
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fd := sys.FdPreopen + 1
tests := []struct {
@@ -2260,7 +2266,8 @@ func Test_fdReaddir_Rewind(t *testing.T) {
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
fd, errno := fsc.OpenFile(fsc.RootFS(), ".", experimentalsys.O_RDONLY, 0)
preopen := getPreopen(t, fsc)
fd, errno := fsc.OpenFile(preopen, ".", experimentalsys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
mem := mod.Memory()
@@ -2307,7 +2314,7 @@ func Test_fdReaddir_Errors(t *testing.T) {
memLen := mod.Memory().Size()
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fileFD, errno := fsc.OpenFile(preopen, "animals.txt", experimentalsys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
@@ -2523,7 +2530,7 @@ func Test_fdRenumber(t *testing.T) {
defer r.Close(testCtx)
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
// Sanity check of the file descriptor assignment.
fileFDAssigned, errno := fsc.OpenFile(preopen, "animals.txt", experimentalsys.O_RDONLY, 0)
@@ -2637,7 +2644,8 @@ func Test_fdSeek_Errors(t *testing.T) {
defer r.Close(testCtx)
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
require.Zero(t, fsc.RootFS().Mkdir("dir", 0o0777))
preopen := getPreopen(t, fsc)
require.Zero(t, preopen.Mkdir("dir", 0o0777))
dirFD := requireOpenFD(t, mod, "dir")
memorySize := mod.Memory().Size()
@@ -3576,10 +3584,12 @@ func Test_pathFilestatSetTimes(t *testing.T) {
sys := mod.(*wasm.ModuleInstance).Sys
fsc := sys.FS()
preopen := getPreopen(t, fsc)
var oldSt sysapi.Stat_t
var errno experimentalsys.Errno
if tc.expectedErrno == wasip1.ErrnoSuccess {
oldSt, errno = fsc.RootFS().Stat(pathName)
oldSt, errno = preopen.Stat(pathName)
require.EqualErrno(t, 0, errno)
}
@@ -3591,7 +3601,7 @@ func Test_pathFilestatSetTimes(t *testing.T) {
return
}
newSt, errno := fsc.RootFS().Stat(pathName)
newSt, errno := preopen.Stat(pathName)
require.EqualErrno(t, 0, errno)
if platform.CompilerSupported() {
@@ -3630,7 +3640,8 @@ func Test_pathLink(t *testing.T) {
newDirPath := joinPath(tmpDir, newDirName)
require.NoError(t, os.MkdirAll(joinPath(tmpDir, newDirName), 0o700))
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
newFd, errno := fsc.OpenFile(fsc.RootFS(), newDirName, 0o600, 0)
preopen := getPreopen(t, fsc)
newFd, errno := fsc.OpenFile(preopen, newDirName, 0o600, 0)
require.EqualErrno(t, 0, errno)
mem := mod.Memory()
@@ -3981,7 +3992,7 @@ func writeAndCloseFile(t *testing.T, fsc *sys.FSContext, fd int32) []byte {
func requireOpenFD(t *testing.T, mod api.Module, path string) int32 {
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fd, errno := fsc.OpenFile(preopen, path, experimentalsys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
@@ -4935,7 +4946,7 @@ func requireOpenFile(t *testing.T, tmpDir string, pathName string, data []byte,
mod, r, log := requireProxyModule(t, wazero.NewModuleConfig().WithFSConfig(fsConfig))
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
fd, errno := fsc.OpenFile(preopen, pathName, oflags, 0)
require.EqualErrno(t, 0, errno)
@@ -4954,7 +4965,7 @@ func Test_fdReaddir_dotEntryHasARealInode(t *testing.T) {
mem := mod.Memory()
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
readDirTarget := "dir"
mem.Write(0, []byte(readDirTarget))
@@ -5007,7 +5018,7 @@ func Test_fdReaddir_opened_file_written(t *testing.T) {
mem := mod.Memory()
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
preopen := fsc.RootFS()
preopen := getPreopen(t, fsc)
dirName := "dir"
dirPath := joinPath(tmpDir, dirName)

11
imports/wasi_snapshot_preview1/wasi_bench_test.go
View File

@@ -12,6 +12,7 @@ import (
"github.com/tetratelabs/wazero/imports/wasi_snapshot_preview1"
"github.com/tetratelabs/wazero/internal/sys"
"github.com/tetratelabs/wazero/internal/testing/proxy"
"github.com/tetratelabs/wazero/internal/testing/require"
"github.com/tetratelabs/wazero/internal/wasip1"
"github.com/tetratelabs/wazero/internal/wasm"
)
@@ -199,7 +200,10 @@ func Benchmark_fdReaddir(b *testing.B) {
// Open the root directory as a file-descriptor.
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
fd, errno := fsc.OpenFile(fsc.RootFS(), ".", experimentalsys.O_RDONLY, 0)
preopenFile, ok := fsc.LookupFile(sys.FdPreopen)
require.True(b, ok)
preopen := preopenFile.FS
fd, errno := fsc.OpenFile(preopen, ".", experimentalsys.O_RDONLY, 0)
if errno != 0 {
b.Fatal(errno)
}
@@ -308,7 +312,10 @@ func Benchmark_pathFilestat(b *testing.B) {
fd := sys.FdPreopen
if bc.fd != sys.FdPreopen {
fsc := mod.(*wasm.ModuleInstance).Sys.FS()
fd, errno := fsc.OpenFile(fsc.RootFS(), "zig", experimentalsys.O_RDONLY, 0)
preopenFile, ok := fsc.LookupFile(sys.FdPreopen)
require.True(b, ok)
preopen := preopenFile.FS
fd, errno := fsc.OpenFile(preopen, "zig", experimentalsys.O_RDONLY, 0)
if errno != 0 {
b.Fatal(errno)
}

5
internal/fstest/fstest.go
View File

@@ -1,5 +1,5 @@
// Package fstest defines filesystem test cases that help validate host
// functions implementing WASI and `GOOS=js GOARCH=wasm`. Tests are defined
// functions implementing WASI. Tests are defined
// here to reduce duplication and drift.
//
// Here's an example using this inside code that compiles to wasm.
@@ -13,9 +13,6 @@
//
// Failures found here should result in new tests in the appropriate package,
// for example, sysfs or wasi_snapshot_preview1.
//
// This package must have no dependencies. Otherwise, compiling this with
// TinyGo or `GOOS=js GOARCH=wasm` can become bloated or complicated.
package fstest
import (

23
internal/sys/fs.go
View File

@@ -251,9 +251,6 @@ func (d *DirentCache) cachedDirents(n uint32) []sys.Dirent {
}
type FSContext struct {
// rootFS is the root ("/") mount.
rootFS sys.FS
// openedFiles is a map of file descriptor numbers (>=FdPreopen) to open files
// (or directories) and defaults to empty.
// TODO: This is unguarded, so not goroutine-safe!
@@ -264,19 +261,6 @@ type FSContext struct {
// descriptors to file entries.
type FileTable = descriptor.Table[int32, *FileEntry]
// RootFS returns a possibly unimplemented root filesystem. Any files that
// should be added to the table should be inserted via InsertFile.
//
// TODO: This is only used by GOOS=js and tests: Remove when we remove GOOS=js
// (after Go 1.22 is released).
func (c *FSContext) RootFS() sys.FS {
if rootFS := c.rootFS; rootFS == nil {
return sys.UnimplementedFS{}
} else {
return rootFS
}
}
// LookupFile returns a file if it is in the table.
func (c *FSContext) LookupFile(fd int32) (*FileEntry, bool) {
return c.openedFiles.Lookup(fd)
@@ -412,19 +396,18 @@ func (c *Context) InitFSContext(
}
c.fsc.openedFiles.Insert(errWriter)
for i, fs := range fs {
for i, f := range fs {
guestPath := guestPaths[i]
if StripPrefixesAndTrailingSlash(guestPath) == "" {
// Default to bind to '/' when guestPath is effectively empty.
guestPath = "/"
c.fsc.rootFS = fs
}
c.fsc.openedFiles.Insert(&FileEntry{
FS: fs,
FS: f,
Name: guestPath,
IsPreopen: true,
File: &lazyDir{fs: fs},
File: &lazyDir{fs: f},
})
}

184
internal/sys/fs_test.go
View File

@@ -11,7 +11,6 @@ import (
gofstest "testing/fstest"
"github.com/tetratelabs/wazero/experimental/sys"
"github.com/tetratelabs/wazero/internal/fstest"
"github.com/tetratelabs/wazero/internal/sysfs"
testfs "github.com/tetratelabs/wazero/internal/testing/fs"
"github.com/tetratelabs/wazero/internal/testing/require"
@@ -63,8 +62,9 @@ func TestNewFSContext(t *testing.T) {
fsc := c.fsc
defer fsc.Close()
rootFs := tc.fs
preopenedDir, _ := fsc.openedFiles.Lookup(FdPreopen)
require.Equal(t, tc.fs, fsc.rootFS)
require.Equal(t, tc.fs, rootFs)
require.NotNil(t, preopenedDir)
require.Equal(t, "/", preopenedDir.Name)
@@ -250,192 +250,18 @@ func TestFSContext_Renumber(t *testing.T) {
})
}
func TestDirentCache_Read(t *testing.T) {
c := Context{}
err := c.InitFSContext(nil, nil, nil, []sys.FS{&sysfs.AdaptFS{FS: fstest.FS}}, []string{"/"}, nil)
require.NoError(t, err)
fsc := c.fsc
defer fsc.Close()
d, errno := sysfs.OpenFSFile(fstest.FS, "dir", 0, 0)
require.EqualErrno(t, 0, errno)
defer d.Close()
testDirents, errno := d.Readdir(-1)
if errno != 0 {
panic(errno)
}
testDirents = append([]sys.Dirent{
{Name: ".", Type: fs.ModeDir},
{Name: "..", Type: fs.ModeDir},
}, testDirents...)
tests := []struct {
name string
initialDir string
dir func(fd int32)
fd int32
pos uint64
n uint32
expectedDirents []sys.Dirent
expectedErrno sys.Errno
}{
{
name: "empty dir has dot entries",
initialDir: "emptydir",
pos: 0,
n: 100,
expectedDirents: testDirents[:2],
},
{
name: "rewind empty directory",
initialDir: "emptydir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 5)
},
pos: 0,
n: 100,
expectedDirents: testDirents[:2],
},
{
name: "full read",
initialDir: "dir",
pos: 0,
n: 100,
expectedDirents: testDirents,
},
{
name: "read first",
initialDir: "dir",
pos: 0,
n: 1,
expectedDirents: testDirents[:1],
},
{
name: "read second",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 1)
},
pos: 1,
n: 1,
expectedDirents: testDirents[1:2],
},
{
name: "read second and third",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 1)
},
pos: 1,
n: 2,
expectedDirents: testDirents[1:3],
},
{
name: "read exactly third",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 2)
},
pos: 2,
n: 1,
expectedDirents: testDirents[2:3],
},
{
name: "read third and beyond",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 2)
},
pos: 2,
n: 5,
expectedDirents: testDirents[2:],
},
{
name: "read exhausted directory",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 5)
},
pos: 5,
n: 5,
expectedDirents: nil,
},
{
name: "rewind directory",
initialDir: "dir",
dir: func(fd int32) {
f, _ := fsc.LookupFile(fd)
rdd, _ := f.DirentCache()
_, _ = rdd.Read(0, 5)
},
pos: 0,
n: 5,
expectedDirents: testDirents,
},
{
name: "DirentCache: not a dir",
initialDir: "dir/-",
pos: 0,
n: 1,
expectedErrno: sys.ENOTDIR,
},
{
name: "pos invalid when no prior state",
initialDir: "dir",
pos: 1,
n: 1,
expectedErrno: sys.ENOENT,
},
}
for _, tt := range tests {
tc := tt
t.Run(tc.name, func(t *testing.T) {
fd, errno := fsc.OpenFile(fsc.RootFS(), tc.initialDir, sys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
defer fsc.CloseFile(fd) // nolint
f, _ := fsc.LookupFile(fd)
dir, errno := f.DirentCache()
if errno != 0 {
require.EqualErrno(t, tc.expectedErrno, errno)
return
}
if tc.dir != nil {
tc.dir(fd)
}
dirents, errno := dir.Read(tc.pos, tc.n)
require.EqualErrno(t, tc.expectedErrno, errno)
require.Equal(t, tc.expectedDirents, dirents)
})
}
}
// This is similar to https://github.com/WebAssembly/wasi-testsuite/blob/ac32f57400cdcdd0425d3085c24fc7fc40011d1c/tests/rust/src/bin/fd_readdir.rs#L120
func TestDirentCache_ReadNewFile(t *testing.T) {
tmpDir := t.TempDir()
c := Context{}
err := c.InitFSContext(nil, nil, nil, []sys.FS{sysfs.DirFS(tmpDir)}, []string{"/"}, nil)
root := sysfs.DirFS(tmpDir)
err := c.InitFSContext(nil, nil, nil, []sys.FS{root}, []string{"/"}, nil)
require.NoError(t, err)
fsc := c.fsc
defer fsc.Close()
fd, errno := fsc.OpenFile(fsc.RootFS(), ".", sys.O_RDONLY, 0)
fd, errno := fsc.OpenFile(root, ".", sys.O_RDONLY, 0)
require.EqualErrno(t, 0, errno)
defer fsc.CloseFile(fd) // nolint
f, _ := fsc.LookupFile(fd)

2
internal/sysfs/oflag_test.go
View File

@@ -49,7 +49,7 @@ func Test_toOsOpenFlag(t *testing.T) {
}
// Example of a flag that can be or'd into O_RDONLY even if not
// currently supported in WASI or GOOS=js
// currently supported in WASI.
const O_NOATIME = sys.Oflag(0x40000)
require.Zero(t, 0, toOsOpenFlag(O_NOATIME))
}

2
internal/sysfs/sysfs.go
View File

@@ -1,5 +1,5 @@
// Package sysfs includes a low-level filesystem interface and utilities needed
// for WebAssembly host functions (ABI) such as WASI and runtime.GOOS=js.
// for WebAssembly host functions (ABI) such as WASI.
//
// The name sysfs was chosen because wazero's public API has a "sys" package,
// which was named after https://github.com/golang/sys.

2
internal/sysfs/sysfs_test.go
View File

@@ -212,7 +212,7 @@ human
// Make sure O_RDONLY isn't treated bitwise as it is usually zero.
t.Run("or'd flag", func(t *testing.T) {
// Example of a flag that can be or'd into O_RDONLY even if not
// currently supported in WASI or GOOS=js
// currently supported in WASI.
const O_NOATIME = experimentalsys.Oflag(0x40000)
f, errno := testFS.OpenFile("animals.txt", experimentalsys.O_RDONLY|O_NOATIME, 0)

2
internal/wasip1/errno.go
View File

@@ -262,7 +262,7 @@ var errnoToString = [...]string{
// ToErrno coerces the error to a WASI Errno.
//
// Note: Coercion isn't centralized in sys.FSContext because ABI use different
// error codes. For example, wasi-filesystem and GOOS=js don't map to these
// error codes. For example, wasi-filesystem doesn't map to these
// Errno.
func ToErrno(errno sys.Errno) Errno {
switch errno {

1
site/content/languages/_index.md
View File

@@ -17,7 +17,6 @@ e.g. If your source is in Go, you might compile it with TinyGo.
Below are notes wazero contributed so far, in alphabetical order by language.
* [Go]({{< relref "/go.md" >}}) e.g. `GOOS=js GOARCH=wasm go build -o X.wasm X.go`
* [TinyGo]({{< relref "/tinygo.md" >}}) e.g. `tinygo build -o X.wasm -target=wasi X.go`
* [Rust]({{< relref "/rust.md" >}}) e.g. `rustc -o X.wasm --target wasm32-wasi X.rs`
* [Zig]({{< relref "/zig.md" >}}) e.g. `zig build-exe X.zig -target wasm32-wasi`

367
site/content/languages/go.md
View File

@@ -1,367 +0,0 @@
+++
title = "Go"
+++
## Introduction
When `GOOS=js GOARCH=wasm`, Go's compiler targets WebAssembly Binary format
(%.wasm).
Here's a typical compilation command:
```bash
$ GOOS=js GOARCH=wasm go build -o my.wasm .
```
The operating system is "js", but more specifically it is [wasm_exec.js][1].
This package runs the `%.wasm` just like `wasm_exec.js` would.
## Experimental
It is important to note that while there are some interesting features, such
as HTTP client support, the ABI (host functions imported by Wasm) used is
complicated and custom to Go. For this reason, there are few implementations
outside the web browser.
Moreover, Go defines js "EXPERIMENTAL... exempt from the Go compatibility
promise." While WebAssembly signatures haven't broken between 1.18 and 1.19,
then have in the past and can in the future.
Due to lack of adoption, support and relatively high implementation overhead,
most choose [TinyGo]({{< relref "/tinygo.md" >}}) to compile source code, even if it supports less
features.
## WebAssembly Features
`GOOS=js GOARCH=wasm` uses instructions in [WebAssembly Core Specification 1.0]
[15] unless `GOWASM` includes features added afterwards.
Here are the valid [GOWASM values][16]:
* `satconv` - [Non-trapping Float-to-int Conversions][17]
* `signext` - [Sign-extension operators][18]
Note that both the above features are included [working draft][19] of
WebAssembly Core Specification 2.0.
## Constraints
Please read our overview of WebAssembly and
[constraints]({{< ref "_index.md#constraints" >}}). In short, expect
limitations in both language features and library choices when developing your
software.
`GOOS=js GOARCH=wasm` has a custom ABI which supports a subset of features in
the Go standard library. Notably, the host can implement time, crypto, file
system and HTTP client functions. Even where implemented, certain operations
will have no effect for reasons like ignoring HTTP request properties or fake
values returned (such as the pid). When not supported, many functions return
`syscall.ENOSYS` errors, or the string form: "not implemented on js".
Here are the more notable parts of Go which will not work when compiled via
`GOOS=js GOARCH=wasm`, resulting in `syscall.ENOSYS` errors:
* Raw network access. e.g. `net.Bind`
* File descriptor control (`fnctl`). e.g. `syscall.Pipe`
* Arbitrary syscalls. Ex `syscall.Syscall`
* Process control. e.g. `syscall.Kill`
* Kernel parameters. e.g. `syscall.Sysctl`
* Timezone-specific clock readings. e.g. `syscall.Gettimeofday`
## Memory
Memory layout begins with the "zero page" of size `runtime.minLegalPointer`
(4KB) which matches the `ssa.minZeroPage` in the compiler. It is then followed
by 8KB reserved for args and environment variables. This means the data section
begins at [ld.wasmMinDataAddr][12], offset 12288.
## System Calls
Please read our overview of WebAssembly and
[System Calls]({{< ref "_index.md#system-calls" >}}). In short, WebAssembly is
a stack-based virtual machine specification, so operates at a lower level than
an operating system.
"syscall/js.*" are host functions for features the operating system would
otherwise provide. These also manage the JavaScript object graph, including
functions to make and finalize objects, arrays and numbers (`js.Value`).
Each `js.Value` has a `js.ref`, which is either a numeric literal or an object
reference depending on its 64-bit bit pattern. When an object, the first 31
bits are its identifier.
There are several pre-defined values with constant `js.ref` patterns. These are
either constants, globals or otherwise needed in initializers.
For example, the "global" value includes properties like "fs" and "process"
which implement [system calls][7] needed for functions like `os.Getuid`.
Notably, not all system calls are implemented as some are stubbed by the
compiler to return zero values or `syscall.ENOSYS`. This means not all Go code
compiled to wasm will operate. For example, you cannot launch processes.
Details beyond this are best looking at the source code of [js.go][5], or its
unit tests.
## Concurrency
Please read our overview of WebAssembly and
[concurrency]({{< ref "_index.md#concurrency" >}}). In short, the current
WebAssembly specification does not support parallel processing.
Some internal code may seem strange knowing this. For example, Go's [function
wrapper][9] used for `GOOS=js` is implemented using locks. Seeing this, you may
feel the host side of this code (`_makeFuncWrapper`) should lock its ID
namespace for parallel use as well.
Digging deeper, you'll notice the [atomics][10] defined by `GOARCH=wasm` are
not actually implemented with locks, rather it is awaiting the ["Threads"
proposal][11].
In summary, while goroutines are supported in `GOOS=js GOARCH=wasm`, they won't
be able to run in parallel until the WebAssembly Specification includes atomics
and Go's compiler is updated to use them.
## Error handling
There are several `js.Value` used to implement `GOOS=js GOARCH=wasm` including
the global, file system, HTTP round tripping, processes, etc. All of these have
functions that may return an error on `js.Value.Call`.
However, `js.Value.Call` does not return an error result. Internally, this
dispatches to the wasm imported function `valueCall`, and interprets its two
results: the real result and a boolean, represented by an integer.
When false, `js.Value.Call` panics with a `js.Error` constructed from the first
result. This result must be an object with one of the below properties:
* JavaScript (GOOS=js): the string property "message" can be anything.
* Syscall error (GOARCH=wasm): the string property "code" is constrained.
* The code must be like "EIO" in [errnoByCode][13] to avoid a panic.
Details beyond this are best looking at the source code of [js.go][5], or its
unit tests.
## Identifying wasm compiled by Go
If you have a `%.wasm` file compiled by Go (via [asm.go][2]), it has a custom
section named "go.buildid".
You can verify this with wasm-objdump, a part of [wabt][3]:
```
$ wasm-objdump --section=go.buildid -x my.wasm
example3.wasm: file format wasm 0x1
Section Details:
Custom:
- name: "go.buildid"
```
## Module Exports
Until [wasmexport][4] is implemented, the [compiled][2] WebAssembly exports are
always the same:
* "mem" - (memory 265) 265 = data section plus 16MB
* "run" - (func (param $argc i32) (param $argv i32)) the entrypoint
* "resume" - (func) continues work after a timer delay
* "getsp" - (func (result i32)) returns the stack pointer
## Module Imports
Go's [compiles][3] all WebAssembly imports in the module "go", and only
functions are imported.
Except for the "debug" function, all function names are prefixed by their go
package. Here are the defaults:
* "debug" - is always function index zero, but it has unknown use.
* "runtime.*" - supports system-call like functionality `GOARCH=wasm`
* "syscall/js.*" - supports the JavaScript model `GOOS=js`
## PC_B calling conventions
The assembly `CallImport` instruction doesn't compile signatures to WebAssembly
function types, invoked by the `call` instruction.
Instead, the compiler generates the same signature for all functions: a single
parameter of the stack pointer, and invokes them via `call.indirect`.
Specifically, any function compiled with `CallImport` has the same function
type: `(func (param $sp i32))`. `$sp` is the base memory offset to read and
write parameters to the stack (at 8 byte strides even if the value is 32-bit).
So, implementors need to read the actual parameters from memory. Similarly, if
there are results, the implementation must write those to memory.
For example, `func walltime() (sec int64, nsec int32)` writes its results to
memory at offsets `sp+8` and `sp+16` respectively.
Note: WebAssembly compatible calling conventions has been discussed and
[attempted](https://go-review.googlesource.com/c/go/+/350737) in Go before.
## Go-defined exported functions
[Several functions][6] differ in calling convention by using WebAssembly type
signatures instead of the single SP parameter summarized above. Functions used
by the host have a "wasm_export_" prefix, which is stripped. For example,
"wasm_export_run" is exported as "run", defined in [rt0_js_wasm.s][7]
Here is an overview of the Go-defined exported functions:
* "run" - Accepts "argc" and "argv" i32 params and begins the "wasm_pc_f_loop"
* "resume" - Nullary function that resumes execution until it needs an event.
* "getsp" - Returns the i32 stack pointer (SP)
## User-defined Host Functions
Users can define their own "go" module function imports by defining a func
without a body in their source and a `%_wasm.s` or `%_js.s` file that uses the
`CallImport` instruction.
For example, given `func logString(msg string)` and the below assembly:
```assembly
#include "textflag.h"
TEXT ·logString(SB), NOSPLIT, $0
CallImport
RET
```
If the package was `main`, the WebAssembly function name would be
"main.logString". If it was `util` and your `go.mod` module was
"github.com/user/me", the WebAssembly function name would be
"github.com/user/me/util.logString".
Regardless of whether the function import was built-in to Go, or defined by an
end user, all imports use `CallImport` conventions. Since these compile to a
signature unrelated to the source, more care is needed implementing the host
side, to ensure the proper count of parameters are read and results written to
the Go stack.
## Hacking
If you run into an issue where you need to change Go's sourcecode, the first
thing you should do is read the [contributing guide][20], which details how to
confirm an issue exists and a fix would be accepted. Assuming they say yes, the
next step is to ensure you can build and test go.
### Make a branch for your changes
First, clone upstream or your fork of golang/go and make a branch off `master`
for your work, as GitHub pull requests are against that branch.
```bash
$ git clone --depth=1 https://github.com/golang/go.git
$ cd go
$ git checkout -b my-fix
```
### Build a branch-specific `go` binary
While your change may not affect the go binary itself, there are checks inside
go that require version matching. Build a go binary from source to avoid these:
```bash
$ cd src
$ GOOS=js GOARCH=wasm ./make.bash
Building Go cmd/dist using /usr/local/go. (go1.19 darwin/amd64)
Building Go toolchain1 using /usr/local/go.
--snip--
$ cd ..
$ bin/go version
go version devel go1.19-c5da4fb7ac Fri Jul 22 20:12:19 2022 +0000 darwin/amd64
```
Tips:
* The above `bin/go` was built with whatever go version you had in your path!
* `GOARCH` here is what the resulting `go` binary can target. It isn't the
architecture of the current host (`GOHOSTARCH`).
### Setup ENV variables for your branch.
To test the Go you just built, you need to have `GOROOT` set to your workspace,
and your PATH configured to find both `bin/go` and `misc/wasm/go_js_wasm_exec`.
```bash
$ export GOROOT=$PWD
$ export PATH=${GOROOT}/misc/wasm:${GOROOT}/bin:$PATH
```
Tip: `go_js_wasm_exec` is used because Go doesn't embed a WebAssembly runtime
like wazero. In other words, go can't run the wasm it just built. Instead,
`go test` uses Node.js which it assumes is installed on your host!
### Iterate until ready to submit
Now, you should be all set and can iterate similar to normal Go development.
The main thing to keep in mind is where files are, and remember to set
`GOOS=js GOARCH=wasm` when running go commands.
For example, if you fixed something in the `syscall/js` package
(`${GOROOT}/src/syscall/js`), test it like so:
```bash
$ GOOS=js GOARCH=wasm go test syscall/js
ok syscall/js 1.093s
```
### Notes
Here are some notes about testing `GOOS=js GOARCH=wasm`
#### Skipped tests
You may find tests are skipped (e.g. when run with `-v` arg).
```
=== RUN TestSeekError
os_test.go:1598: skipping test on js
```
The go test tree has functions to check if a platform is supported before
proceeding. This allows incremental development of platforms, or avoids things
like launching subprocesses on wasm, which won't likely ever support that.
#### Filesystem access
`TestStat` tries to read `/etc/passwd` due to a [runtime.GOOS default][21].
As `GOOS=js GOARCH=wasm` is a virtualized operating system, this may not make
sense, because it has no files representing an operating system.
Moreover, as of Go 1.19, tests don't pass through any configuration to hint at
the real OS underneath the VM. You might suspect running wasm tests on Windows
would fail, as that OS has no `/etc/passwd` file. In fact, they would except
Windows tests don't pass anyway because the script that invokes Node.JS,
[wasm_exec_node.js][22], doesn't actually work on Windows.
```bash
$ GOOS=js GOARCH=wasm go test os
fork/exec C:\Users\fernc\AppData\Local\Temp\go-build2236168911\b001\os.test: %1 is not a valid Win32 application.
FAIL os 0.034s
FAIL
```
Hosts like Darwin and Linux pass these tests because they include files like
`/etc/passwd` and the test runner (`wasm_exec_node.js`) is configured to pass
through any file system calls without filtering. Specifically,
`globalThis.fs = require("fs")` allows code compiled to wasm any file access
the host operating system's underlying controls permit.
[1]: https://github.com/golang/go/blob/go1.20/misc/wasm/wasm_exec.js
[2]: https://github.com/golang/go/blob/go1.20/src/cmd/link/internal/wasm/asm.go
[3]: https://github.com/WebAssembly/wabt
[4]: https://github.com/golang/proposal/blob/6130999a9134112b156deb52da81a3cf219a6509/design/42372-wasmexport.md
[5]: https://github.com/golang/go/blob/go1.20/src/syscall/js/js.go
[6]: https://github.com/golang/go/blob/go1.20/src/cmd/internal/obj/wasm/wasmobj.go#L796-L810
[7]: https://github.com/golang/go/blob/go1.20/src/runtime/rt0_js_wasm.s#L17-L21
[8]: https://github.com/golang/go/blob/go1.20/src/syscall/syscall_js.go#L292-L306
[9]: https://github.com/golang/go/blob/go1.20/src/syscall/js/func.go#L41-L51
[10]: https://github.com/golang/go/blob/go1.20/src/runtime/internal/atomic/atomic_wasm.go#L5-L6
[11]: https://github.com/WebAssembly/proposals
[12]: https://github.com/golang/go/blob/go1.20/src/cmd/link/internal/ld/data.go#L2457
[13]: https://github.com/golang/go/blob/go1.20/src/syscall/tables_js.go#L371-L494
[15]: https://www.w3.org/TR/2019/REC-wasm-core-1-20191205/
[16]: https://github.com/golang/go/blob/go1.20/src/internal/buildcfg/cfg.go#L136-L150
[17]: https://github.com/WebAssembly/spec/blob/wg-2.0.draft1/proposals/nontrapping-float-to-int-conversion/Overview.md
[18]: https://github.com/WebAssembly/spec/blob/wg-2.0.draft1/proposals/sign-extension-ops/Overview.md
[19]: https://www.w3.org/TR/2022/WD-wasm-core-2-20220419/
[20]: https://github.com/golang/go/blob/go1.20/CONTRIBUTING.md
[21]: https://github.com/golang/go/blob/go1.20/src/os/os_test.go#L109-L116
[22]: https://github.com/golang/go/blob/go1.20/misc/wasm/wasm_exec_node.js