Unsafe functions such as `unsafe.Alignof`, `unsafe.Offsetof` and `unsafe.Sizeof` can be used for type declarations early on during compile, and as such, must be treated as builtins returning constants at compile time. It is still necessary to explicitely enable unsafe support in yaegi.
The support of `unsafe.Add` has also been added.
Fixes#1544.
In the case of a Go short definition (i.e. `a, b := f()`), the new defined variables must be (re-)created in order to preserve the previous value (if in a loop) which can be still in use in the context of a closure. This must not apply to redeclared variables which simply see their value reassigned.
The problem was both occuring in callBin() for runtime functions and assignFromCall() for functions created in the interpreter.
Fixes#1497.
When generating a new type, the parameter type was not correctly duplicated in the new AST. This is fixed by making copyNode recursive if needed. The out of order processing of generic types has also been fixed.
Fixes#1488
This change implements a workaround to better support composed
interfaces in yaegi and let the interpreter define objects which
implement multiple interfaces at once.
We use the existing MapTypes to store what possible composed interface
wrapper could be used for some interfaces. When generating an interface
wrapper, the wrapper with the highest number of implemented methods is
chosen.
This is still an imperfect solution but it improves the accuracy of
interpreter in some critical cases.
This workaround could be removed in future if/when golang/go#15924
is resolved.
Fixes#1425.
Status:
* [x] parsing code with generics
* [x] instantiate generics from concrete types
* [x] automatic type inference
* [x] support of generic recursive types
* [x] support of generic methods
* [x] support of generic receivers in methods
* [x] support of multiple type parameters
* [x] support of generic constraints
* [x] tests (see _test/gen*.go)
Fixes#1363.
In addition:
- extract commmand now skips exported generics in runtime wrappers
- interp_consistent_test.go fixed for go1.18 and go1.19
- move minimal version of go compiler to go1.18
Note that this version is incompatible with go1.17 and before due
to the major changes in the stdlib go parser.
To be merged once go1.19 is officially released (not before).
We finally address a long standing limitation of the interpreter:
the capacity to generate the correct interface wrapper for an
anonymous interface{} function parameter of a binary function.
It allows for example fmt.Printf to invoke the String method
of an object defined within the interpreter, or json.Marshal
to invoke a textMarshaler method if it exists and if there is
no Marshaler method already defined for the passed interpreter
object.
To achieve that, we add a new mapType part of the "Used" symbols
to describe what not empty interfaces are expected and in which
priority order. This information can not be guessed and is found
in the related package documentation, then captured in stdlib/maptypes.go.
Then, at compile time and/or during execution, a lookup on mapTypes
is performed to allow the correct wrapper to be generated.
This change adds a new MapType type to the stdlib package.
Fixes#435.
In restricted mode, replace environment related symbols in
stdlib os package by a version which operates on a private copy
per interpreter context.
It allows to have concurrent interpreters in the same process
operating each in their own environment without affecting each
other or the host.
If unrestricted opt is set, this behaviour is disabled, and the
default symbols from stdlib are used.
Note also that no modification is done for syscall package, as it
should be not used in restricted mode.
In some cases, we use many yaegi instances in one process, each interpreter will expect its own os.Args. This is done in interp.go as an option in interp.Options besides Stdin Stdout and Stderr.
Adds CompileAST, which can be used to compile Go AST directly. This
allows users to delegate parsing of source to their own code instead of
relying on the interpreter.
CLoses#1251
When a define statement relies on a selector or type that may exist in another file it should revisit once GTA is complete. This allows that revisit.
**Note:** In order to keep the original GTA error for the define statement, so the error received is correct and meaningful, I have added a node `meta` property. I decided to make it generic as it may be useful in future. There may be a better way to stash errors across the GTA runs and am open to suggestion here.
Fixes#1253
It was initially assumed that `nodeType` needed to rebuild the type from scratch, but this is not the case anymore. The existing type constructors are now used in `nodeType` to make it more readable. In doing this some bugs in type strings were found and fixed, along with adding the real package name to the type.
As `ptrOf` is now the only place that pointer types are constructed, it is feasible to cache the pointer type on the value type. To ensures that we have a consistent pointer type for any value type.
This adds `itype.str` which is a string representation of the type built when the type is built. The goal is to make type comparison simpler and centralise the creation of types just to constructors and `nodeType`. `nodeType` continues to build types in parts so to reuse underlying types better.
Make use of fs.FS (new to go 1.16) to allow for reading source files from diverse filesystems (local, embed, custom).
* `Options` has a new field `SourcecodeFilesystem fs.FS` so users can supply their own read-only filesystem containing source code.
* Defaults to the local filesystems (via `RealFS` - a thin `os.Open` wrapper complying with `fs.FS`) so regular users should see no change in behaviour.
* When no filesystem is available (e.g. WASM, or if you want to embed files to retain single binary distribution) an alternative filesystem is preferable to using `Eval(string)` as that requires the stringy code to be a single file monolith instead of multiple files. By using an `fs.FS` we can use `EvalPath()` and gain the ability to handle multiple files and packages.
* You can make use of embed filesystems (https://pkg.go.dev/embed) and custom filesystems obeying the `fs.FS` interface (I use one for http served zip files when targeting wasm as there is no local filesystem on wasm). Tests can make use of `fstest.Map`.
* NOTE: This does NOT affect what the running yaegi code considers its local filesystem, this is only for the interpreter finding the source code.
See `example/fs/fs_test.go` for an example.
Fixes#1200.
Add getConcreteType to retrieve the concrete type of a nested interface
value implementing a specific interface for which a wrapper exists.
If method resolution fails at runtime, a panic is now issued instead
of an error message and continue.
Fixes#1187.
Store the interpreter value of the interface object to wrap as
a field called IValue, at offset 0 in wrapper structures.
Update extract to include IValue field.
In typeAssert, detect interface wrapper, and dereference the
interpreter value from IValue wrapper field.
Fixes#1166.
### Background
#1102 changed how `Interpreter.Use` interprets export paths such that the last path component is stripped and used as the package name. This resulted in #1139 - attempting to Use an export with only one path component, such as `foo`, would result in the import path being `.`.
### Breaking API Change
This PR changes the signature of `Interpreter.Use` from `Use(Exports)` to `Use(Exports) error`.
### Fix for #1139
With this PR, if Use is called with an incomplete export path, such as `foo`, Use will return an error.
Use YAEGI_SPECIAL_STDIO env boolean to overwrite os.Stdxxx by
non file descriptors io.Writer / io.Reader interfaces. It is set
to true when testing to allow redirection to byte buffers.
The default behaviour is now to preserve the original concrete type
when sandboxing stdio, which maintains compatibility.
Fixes#1092.
Some binary method calls were wrongly rejected. There is still
some ambiguous cases as binary method signature may include or
not the receiver as first argument, depending on how the method
was resolved.
With this fix, `import "golang.org/x/net/html"` doesn't panic
anymore, but not all tests are passing yet, i.e.
`yaegi test golang.org/x/net/html` still has failures, to be
investigated.
Fixes#1107.
The heuristic to generate a package name identifier was incorrect. Now for binary packages, the package identifier is obtained by a symbol, generated by extract, which contains the string argument of package statement in source file. This should ensure an always correct default package identifier.
Fixes#1095.
This feature was already present, but part of REPL only.
It's now also possible to apply it when evaluating a string
(-e flag). Default package names collision handling is no
longer hard-coded.
With -e flag, the eval result is now printed if valid, allowing
simpler commands:
yaegi -e 'reflect.TypeOf(fmt.Printf)'
instead of:
yaegi -e 'println(reflect.TypeOf(fmt.Printf))'
Fixes#1084.
The interpreter is exposed to itself through a "Self" var which
is set on "Use" of the interpreter package.
It allows meta-programming features, for example using "Eval" in
the current interpreter context, or enabling self-inspection
capabilities.
The first change forces a variable definition to reallocate a
new memory slot to avoid corrupting a previously defined one in
a loop block.
The second change ensures that the frame clone operations obtains
a copy of the original data slice, to preserve the original context
set in a loop.
Fixes#1035.
As mentioned in #1030, when an Eval panic, it is print with `fmt.Println()` and not to the configured `interp.Options.Stderr`. According to https://github.com/traefik/yaegi/blob/master/interp/interp.go#L210, it should be removed in future version so I'm not sure if this pull request is necessary. However, it could fix the issue in the meanwhile.
Fixes#1030.
In some cases, the global character of a value was lost, leading to
undefined behaviour. Now a node level field of -1 means that the value
is global, and that it should be accessed from the root data frame.
Fixes#993.
Often enough when debugging, one does not know exactly what argument
should be given to Symbols, as it's not always clear in which
scope/namespace the symbol one is looking for is stored in.
Therefore, this change enables the Symbols method to now take the empty
string as an argument, which results in all the known symbols to be
returned (keyed by import path).
As a consequence, when an non-empty argument is given, the returned
result should be similar to what we had before, except it is now
returned as the sole entry of an encompassing map.
In addition, the "binary" symbols (i.e. the ones ingested through a
Use call), are now also taken into account.
Since the introduction of restricted stdlib and syscall symbols, the
capability of yaegi to interpret itself was broken.
The use of unrestricted symbols is now also controlled by environment
variables, to allow propagation accross nested interpreters.
The interpreter Panic symbol was not wrapped, this is fixed now.
the import path resolution was failing if the working directory was
outside of GOPATH.
The documentation and readme have been ajusted.
Fixes#890.
When working with an untyped const expression involving a division, if
the default type of the result should be an int (for example because the
default types of all the operands are ints as well), then we should make
sure that the operation that is applied is indeed an integer division,
and that the type of the result is not a float.
This is achieved by using the QUO_ASSIGN operator, instead of the QUO
operator.
This should fix several problems lurking around, and it notably fixes
one of the visible consequences, which is a systematic panic when using
the REPL as a "calculator".
This incidentally also allows us to revert what was done in
5dfc3b86dc since it now turns out it was
just a hack to fix one of the symptoms.
Fixes#864
This change allows the interpreter to execute tests and benchmarks
functions provided by packages.
The test subcommand is similar to the "go test" command and
all the relevant flags have been kept.
The ability to evaluate a directory or a package has also been added.
A new method Symbol to access exported symbol values of an interpreted
package has been added. This method is used by the test subcommand.
An EvalTest method has been added to evaluate all Go files, including "*_test.go".
The testing packages from the standard library have been added to stdlib used
symbols.
This change fixes two distinct data races:
1) some global vars of type *itype of the interp package are actually
mutated during the lifecycle of an Interpreter. Even worse: if more than
one Interpreter instance are created and used at a given time, they are
actually racing each other for these global vars.
Therefore, this change replaces these global vars with generator
functions that create the needed type on the fly.
2) the symbols given as argument of Interpreter.Use were directly copied
as reference (since they're maps) when mapped inside an Interpreter
instance. Since the usual case is to give the symbols from the stdlib
package, it means when the interpreter mutates its own symbols in
fixStdio, it would actually mutate the corresponding global vars of the
stdlib package. Again, this is at least racy as soon as several
instances of an Intepreter are concurrently running.
This change fixes the race by making sure Interpreter.Use actually
copies the symbol values instead of copying the references.
The goal is to provide greater control of input, output and error
streams of the interpreter. It is now possible to specify those
as options when creating a new interpreter. The provided values
are propagated to relevant stdlib symbols (i.e fmt.Print, etc).
Care is taken to not update the global variables os.Stdout, os.Stdin
and os.Stderr, as to not interfere with the host process.
The REPL function is now simplified. The deprecated version is removed.
The tests are updated to take advantage of the simplified access
to the interpreter output and errors.
Fixes#752.
In interactive mode, a line starting with the "func" keyword is usually
"wrapped", by prepending to it a "package main" statement, to make it
a valid piece of Go source code.
However, when the line is actually an anonymous function call, such as:
func() { println(3) }()
then this wrapping is not enough, as this is not valid Go in the global
context. Therefore, this kind of of expression must also be wrapped
inside a main func (as is the default case for most REPL inputs).
Since the detection and handling of such a case turned out to be quite
unelegant, this PR instead introduces a retrying phase when a parsing
error occurs for a particular class of cases. That is to say, when a
"func expression" wrapped in a main package fails to be parsed, it is
then wrapped in a main func before parsing is retried.
N.B. TestEvalScanner has been disabled for this change, because the additional test cases revealed a (most-likely already existing) data race.
Fixes#721
The input scanning is now performed in a sub goroutine and
the interrupt is listened in another goroutine, either to cancel Eval
or to cancel the current line scan.
Entering a '\n' after a 'Ctrl-C` to get the prompt is
not necessary anymore.
The recent changes that added some redeclaration checks implicitly added more
strictness related to namespaces and scopes which, among other things, broke
some uses that "accidentally" used to work.
For example, given
const script1 = `
import "fmt"
// more code
`
const script2 = `
import "fmt"
// some other code
`
If one Evals script1, then script2, with the same interpreter, without
specifying any scope, as the two fragments would be considered part of the same
(.go file) scope by default, a redeclaration error would be triggered because
import "fmt" is seen twice.
A work-around would have been to specify (a different) i.Name before each Eval
call, so that each script is considered as coming from a different .go file, and
hence are respectively in different scopes with respect to imports.
That lead us to realize we had to make specifying things such as file-scope, and
"incremental mode" (aka REPL), more obvious in the context of an Eval call.
In addition, we want to lay down the foundations for Yaegi being able to behave
more like the go tool wrt to various inputs, i.e. it should be able to take a
package directory, or an import path, as input, instead of just a .go file.
Hence the introduction of a new kind of Eval method (whose signature is not fixed yet):
func (interp *Interpreter) EvalPath(path string) (res reflect.Value, err error)
It partially solves the problem described above because:
1. the path given to EvalPath can be used as the file-scope hint mentioned
above, for now (even though the related implementation details might change).
2. Eval always runs in incremental mode, whereas EvalPath always runs in
non-incremental mode, hence clarifying the situation in that respect.
And to avoid confusion, the Name field of Interpreter is now non-exported,
since it is somewhat redundant with the path argument of EvalPath.
Note that #731 is not fully fixed (and might never be), as a requirement of the
proposed solution is to move the input bits of code into respective files
(instead of leaving them as strings).
Finally, some related bugfixes, documention changes, and some refactoring have
been included. Notably, there is no "empty scope" anymore, i.e. name defaults
to "_.go" when it is not specified.
Updates #731Fixes#778Fixes#798Fixes#789
Co-authored-by: Marc Vertes <mvertes@free.fr>
Avoid goroutines leak, accumulation of defered functions and
spurious resets of signal handlers. Effectively catch interrupt
signal (Ctrl-C) to cancel current eval.
Fixes#713.
So far the REPL loop was treating any parsing error coming from
go/parser to generate the AST, as having occurred because the source
code was not yet complete (unfinished block). And it was therefore
ignoring all of them.
However, some of these errors are legitimate, and must be caught as soon
as they occur, otherwise the REPL cycle would stay in an errored state
forever (even when the block terminates), without the user getting any
feedback about it.
Therefore, this change adds an extra check when a parsing error occurs,
i.e. it verifies that it looks like an "EOF" error (unfinished block)
before it ignores it (as the user is supposed to terminate the block
eventually). Otherwise the error is treated just like a "non-parsing"
(cfg, gta, ...) error and printed out.
Fixes#637
* fix: constant definition loop on out of order vars
* fix: do not wire global varDecl
* fix: wire and execute global vars
* chore: add tests
* fix: refactor and lint
