Commit Graph

756 Commits

Author SHA1 Message Date
Takeshi Yoneda
c51a63058e Removes importedFn (#926)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-15 15:00:22 +09:00
Takeshi Yoneda
fafcb1cfee Removes unused FunctionInstance.LocalTypes (#925)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-15 14:40:09 +09:00
Crypt Keeper
5f7467b3e0 Enables debug info (a.k.a. DWARF) by default (#924)
This makes DWARF enabled by default, with an opt-out flag. This didn't
need to be experimental as the feature was requested for a long time and
there's no API impact to using it.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
v1.0.0-pre.5
2022-12-15 14:03:20 +09:00
Crypt Keeper
126bd9050d Removes context parameter from instruction-scoped operations (#923)
We originally had a `context.Context` for anything that might be
traced, but it turned out to be only useful for lifecycle and host functions.

For instruction-scoped aspects like memory updates, a context parameter is too
fine-grained and also invisible in practice. For example, most users will use
the compiler engine, and its memory, global or table access will never use go's
context.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-14 16:08:07 +09:00
Crypt Keeper
0ed4002549 Removes experimental.WithFS (#922)
This removes the ability to override the current file system with Go
context, allowing us to simplify common paths and improve performance.

The context override was only used once in GitHub, in Trivy, and we
found another way to do that without it.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-14 15:00:17 +09:00
Takeshi Yoneda
9b503fba6d std.fs.wasi.preopensAlloc is no longer needed (#920)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-13 21:22:17 +09:00
Crypt Keeper
2e13f57f56 wasi: enforce fd_preXXX functions only work on pre-opened files (#919)
At the moment, the only pre-open support we have is the file system
itself (root a.k.a. / or file-descriptor 3). We may in the future add
the ability to pre-open sockets, but in any case, this is where we are
today.

This change hardens logic around fd_preXXX functions, ensuring they only
work on actual pre-opens. This also fixes the path returned in filestat
as we sometimes returned a full path, when typically the basename is the
only part that can be returned.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-13 16:42:53 +09:00
Crypt Keeper
3b70504f3f wasi: ensure someone can tell if root is a real file (#918)
`os.DirFS` ironically doesn't implement `fs.ReadDirFS`, so we cannot
open a directory this way. This blindly attempts to open "." regardless
of if the `fs.FS` implementation is a `fs.ReadDirFS` or not, and if
successful, enforces that the file returned is a directory. If not, a
fake directory is returned.

Doing so allows real stat to be returned for root, and also a chance to
know if a filesystem configured is real or not. Later, we'll need this
to implement open flags.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-13 13:50:00 +09:00
Takeshi Yoneda
7bd1c1600f dwarf: fixes for non-exact matched lines (#916)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-13 13:37:40 +09:00
Crypt Keeper
ce62d77ef0 wasi: allows stat on root file descriptor (3) (#917)
Before, we didn't allow a real stat on the root file descriptor. Now,
those that pass fs.ReadDirFS will return the stat of the root file
(which is implemented by a open against ".").

This also simplifies logic as we always have a file representing root,
even if faked.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-13 11:00:05 +09:00
Crypt Keeper
76bc1f2b9f wasi: enforces oflag dir when opening a directory (#915)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-12 20:58:11 +09:00
Crypt Keeper
3dabad8c16 wasi: skips fd allocation on path_filestat_get and backfills benchmarks (#914)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-12 16:17:29 +09:00
Takeshi Yoneda
45aa798a55 dwarf: include inlined function calls in stack traces (#912)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-12 11:20:14 +09:00
Takeshi Yoneda
97cefe0e2d cmd: enable DWARF by default (#903)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-12 09:07:00 +09:00
Takeshi Yoneda
bc37957839 Upgrade Zig to the latest (#913)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-12 08:40:28 +09:00
Takeshi Yoneda
26bd24996e dwarf: ensure Goroutine-safety in DWARFLines.Line function (#911)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-12 08:22:35 +09:00
Crypt Keeper
e49995cdd3 wasi: benchmarks fd_readdir for both real and fake file systems (#910)
It is likely we'll have to special-case real files in wasi, as wasi-libc
makes behavioral choices based on presence of non-zero inode data. This
adds a defensive benchmark, so that we know what our performance base
case is. This adds both real and fake file systems as both patterns are
in use today.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-12 07:53:56 +09:00
Crypt Keeper
a9f402d840 wasi: optimize and benchmark fd_read (#909)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-09 17:28:39 +09:00
Crypt Keeper
51062b785b wasi: improves fd_write perf when not discarded (#904)
```bash
$ go test -run='^$' -bench '^Benchmark_fdWrite$' -count=5 > old.txt
$ go test -run='^$' -bench '^Benchmark_fdWrite$' -count=5 > new.txt
$ benchstat old.txt new.txt
name                    old time/op  new time/op  delta
_fdWrite/io.Writer-16    138ns ± 1%   132ns ± 3%  -3.95%  (p=0.008 n=5+5)
_fdWrite/io.Discard-16   137ns ± 6%   139ns ± 4%    ~     (p=0.421 n=5+5)
```

Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-09 12:40:02 +09:00
Clifton Kaznocha
3ec5928a83 Simplify namespace (#906)
Signed-off-by: Clifton Kaznocha <ckaznocha@users.noreply.github.com>
Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-09 09:50:48 +09:00
Takeshi Yoneda
fccbc7b2aa dwarf: fixes for unordered addresses of DWARF line entries (#905)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-09 09:12:33 +09:00
Takeshi Yoneda
cc2e6c0697 ci: do not cancel other workflows (#908)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-09 08:40:45 +09:00
Takeshi Yoneda
f001c8ce3a ci: fixes group for main branch commit trigger (#907)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-09 08:34:34 +09:00
Crypt Keeper
6fce8b4f84 wasi: proxy functions to that natural results are visible (#890)
Right now, developing and troubleshooting wasi host functions are difficult as they are required to be WebAssembly 1.0 compatible. Specifically, they can only return a single result, and that's always used for errno. Other results are passed as out-parameters (memory offsets to write results), and their values are not very useful in practice.

This change proxies functions so that we can see their results while still keeping compatibility with wasm signatures. It uses the same approach we use in gojs (`GOOS=js GOARCH=wasm go build...`). What happens is we proxy in wasm, so that the go function implementing the ABI function has natural multi-value returned.

Specifically, we get logging like this, which is better as you can see the values returned:
```diff
-       ==> wasi_snapshot_preview1.args_sizes_get(result.argc=1,result.argv_len=6)
-       <== ESUCCESS
+       --> wasi_snapshot_preview1.args_sizes_get(result.argc=1,result.argv_len=6)
+               ==> wasi_snapshot_preview1.args_sizes_get.go()
+               <== (argc=2,argv_len=5,ESUCCESS)
+       <-- ESUCCESS
```

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-08 16:12:11 +09:00
Crypt Keeper
3423eee0fe Makes log listener default to signed interpretation (#902)
This changes the log listener to assume a value is signed instead of
unsigned, as it leads to logs looking more correct than the other way
around. Notably, values that are often negative, such as seek offset,
look better without a lot of downside.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-08 12:51:26 +09:00
Takeshi Yoneda
ae0c451505 ci: cancel previous runs on the same PRs (#901)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-08 12:39:45 +09:00
Clifton Kaznocha
3b76d699e3 Reduce lock contention during module close (#900)
Signed-off-by: Clifton Kaznocha <ckaznocha@users.noreply.github.com>
Co-authored-by: Clifton Kaznocha <ckaznocha@users.noreply.github.com>
2022-12-08 12:37:11 +09:00
Takeshi Yoneda
7221ded05d wasi: adds Zig stdlib test case for (#882)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-08 12:28:28 +09:00
Robbert van Ginkel
268300a72e Use shifted iota for wasiFdflags values (#899)
Signed-off-by: Robbert van Ginkel <rvanginkel@buf.build>
2022-12-08 10:41:02 +09:00
Takeshi Yoneda
0dde445074 compiler: make moduleContext.functions non-pointer slice (#898)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-07 15:20:24 +09:00
Crypt Keeper
78ac85cce1 wasi: implements fd_fdstat_get and adds stdio support to fd_filestat_get (#895)
This implements fd_fdstat_get and furthers implementation of
fd_fdstat_get. Notably, this does the following:

* Ensures stdio are treated as character devices.
* Ensures someone can stat the root FD (4).
* Fixes encoding as we didn't clear zeros when encoding numbers.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-07 14:28:34 +09:00
Takeshi Yoneda
a129cd7fd5 Removes ModuleInstance.TypeIDsIndex to reduce allocations (#897)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-07 13:49:12 +09:00
Takeshi Yoneda
8294521ec2 Deletes unused ModuleInstance.Types (#896)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-07 12:45:19 +09:00
Takeshi Yoneda
4a60a8f4eb ModuleInstance.Functions/Exports as non-pointer slice (#894)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-07 12:45:06 +09:00
Anuraag Agrawal
eac29c9029 Add a note about the offset requirement in moduleEngine struct (#893)
Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>
2022-12-06 17:12:13 +09:00
Anuraag Agrawal
5de9c3b4bc Store function objects in contiguous memory in heap (#892)
Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>
2022-12-06 17:07:16 +09:00
Takeshi Yoneda
29e4f46d4a Deletes the drifted Deeper dive section in REAMDE (#891)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-06 16:30:59 +09:00
Takeshi Yoneda
65b2431b37 fuzz: enable imports in differential fuzzing (#886)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-06 16:05:16 +09:00
Takeshi Yoneda
b8adb361e8 Fixes FuncRef global initialization with imported globals (#888)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-06 14:32:12 +09:00
Crypt Keeper
af8105ba0e Adds ResultNames to HostFunctionBuilder and FunctionDefinition (#887)
This adds ResultNames to HostFunctionBuilder and FunctionDefinition
which helps for multi-results or special-cased ones.

End users can access result names in `FunctionDefinition.ResultNames` or
set for their own host functions via
`HostFunctionBuilder.WithResultNames`. This change adds them for all
built-in functions where result names help.

Most notably, GOOS=js uses `ProxyFunc` to allow logging when a function
returns multiple results. Before, the results were returned without
names: e.g. `11231,1` and now they are named like `n=11231,ok=1`.

We soon plan to allow more visibility in WASI, for example, logging
results that will write to memory offsets. This infrastructure makes it
possible to do that.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-06 14:30:55 +09:00
Takeshi Yoneda
6c4dd1cfd9 Adds support for DWARF based stack traces (#881)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-05 14:59:45 +09:00
Crypt Keeper
6f30a42828 wasi: optimizes args/environ parsing (#885)
While most compilers will only read args/environ once, tools like WAGI
make heavy use of environment, possibly dozens of long variables. This
optimizes both args and environ for this reason and also to setup for
optimizing other functions.

Here are the notable changes:
* eagerly coerce to byte slices instead of strings
* re-use null terminated length for writing values
* avoid loops that call mem.WriteXXX internally

Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-05 11:07:51 +08:00
Takeshi Yoneda
dca3bfa683 Delete obsolete integration test from wat parser (#884)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-05 12:04:20 +09:00
Takeshi Yoneda
0e9af73038 validation: not panic with redundant Else instrs (#883)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-05 09:04:42 +09:00
Crypt Keeper
e717d09bb7 Adds recently added WASI functions to the website (#880)
Signed-off-by: Adrian Cole <adrian@tetrate.io>
2022-12-02 11:08:26 +08:00
Takeshi Yoneda
e9de52c5d6 fuzz: adds validation target for maybe invalid module compilation (#879)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-02 10:56:13 +09:00
Clifton Kaznocha
dbb13117c2 Fix typos in examples (#878) 2022-12-02 08:44:32 +09:00
Takeshi Yoneda
40e698e068 binary: adds custom section decoder (#877)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-01 20:08:40 +09:00
Takeshi Yoneda
4adfa48eb0 ci: fixes cache key on Minmal Fuzzing (#876)
Signed-off-by: Takeshi Yoneda <takeshi@tetrate.io>
2022-12-01 14:51:08 +09:00
Crypt Keeper
b4f3371a16 wasi: adds fd_readdir (#865)
This adds an implementation of `fd_readdir` for WASI, which ensures a
very large directory is not kept in host memory until its directory is
closed.

Original implementation and test data are with thanks from @jerbob92.

Signed-off-by: Adrian Cole <adrian@tetrate.io>
Co-authored-by: Takeshi Yoneda <takeshi@tetrate.io>
Co-authored-by: jerbob92 <jerbob92@users.noreply.github.com>
v1.0.0-pre.4
2022-12-01 10:19:21 +08:00