When running GTA, the type `path` was set to `rpath`. This equates to the package path (`github.com/traefik/yaegi`) in most cases. In the vendored case the `rpath` is the sub package path `something/vendor/github.com/traefik/yaegi` causing issues in typecheck and likely further down the line. By using the `importPath` it makes this consistent.
**Note:** I have no clue how to test this decently. I am open to options here.
Potentially Fixes#916
Interpreter function types are represented internally by the AST node
of their definition. The conversion operation creates a new node with
the type field pointing to the target type.
Fixes#936.
A non-constant shift expression can be untyped, requiring to apply a
type from inherited context. This change insures that such context is
propagated during CFG pre-order walk, to be used if necessary.
Fixes#927.
The execution flow is such that a node can end up running several chained exec
funcs, some of which actually originate from other nodes. For example, in:
var m []int // L0
println("hello world") // L1
m[0] = 1 // L2
the offending code is on a node on line 2 (out of range error). However, since
the assignment to m is part of the execution flow of the variable m, we'll get
the panic when running all the chained exec funcs attached to the node for m on
line 0.
Which is why, when that happens, we need to look for the actual node (the one on
L2) where the offending instruction originates from, in order to
properly report the origin of the panic.
Fixes#546
The case of a constant arithmetic expression being of float kind because
of quotient was not handled correctly. Simplify constant extraction to
reflect.Value first, then conversion to target type, and let reflect Convert
method panic if conversion is not valid.
Fixes#920.
Type checking on shift operands was failing for untyped variable values.
Fix propagation of type in assignment. Optimize assignment of arithmetic
operations on variables by skipping the assign and writing directly to
destination frame value in the operator function.
Skip some slow tests when given -short test option.
Fixes#917.
Type check was failing for expression such as: `&(*tree)[node:][0]`, as in
```go
tree, _ := huffmanTreePool.Get().(*[]huffmanTree)
...
initHuffmanTree(&(*tree)[node:][0], histogram[l], -1, int16(l))
```
see c3da72aa01/brotli_bit_stream.go (L469)
The assignable check used to be too strict as it lacked the property that
if an untyped const can be represented as a T, then it is assignable to T.
And we can now use that fixed check to add a missing check: in a return
statement, we now make sure that any of the returned elements are
assignable to what the signature tells us they should be.
In unary constant operations, the test for unsigned was defeated by
testing for int first, which is true also for unsigned. Make sure that
testing for unsigned precedes testing for int.
Fixes#907.
This applies to -syscall, -unsafe and -unrestricted flags with the
corresponding env variables YAEGI_SYSCALL, YAEGI_UNSAFE and
YAEGI_UNRESTRICTED, already used in the same way for the run
sub-command.
Often enough when debugging, one does not know exactly what argument
should be given to Symbols, as it's not always clear in which
scope/namespace the symbol one is looking for is stored in.
Therefore, this change enables the Symbols method to now take the empty
string as an argument, which results in all the known symbols to be
returned (keyed by import path).
As a consequence, when an non-empty argument is given, the returned
result should be similar to what we had before, except it is now
returned as the sole entry of an encompassing map.
In addition, the "binary" symbols (i.e. the ones ingested through a
Use call), are now also taken into account.
There was several issues:
- access to field on pointer to struct from runtime: fix in
lookupBinField
- assign operation was skipped when performed in a comm clause
- the direction of comm clause was wrong if a channel send operation was
performed in a body of a receive comm clause
Fixes#884.
Check first for runtime defined array (typ.cat of valueT)
to avoid checking inexisting properties (an panic), when
deciding to use appendSlice or not.
Fixes#880.
Since the introduction of restricted stdlib and syscall symbols, the
capability of yaegi to interpret itself was broken.
The use of unrestricted symbols is now also controlled by environment
variables, to allow propagation accross nested interpreters.
The interpreter Panic symbol was not wrapped, this is fixed now.
the import path resolution was failing if the working directory was
outside of GOPATH.
The documentation and readme have been ajusted.
Fixes#890.
When the type is implicit, the first element in the list of children is
not the type of the composite literal, but it is one of the actual
children, so it should not be discarded.
Fixes#862
When working with an untyped const expression involving a division, if
the default type of the result should be an int (for example because the
default types of all the operands are ints as well), then we should make
sure that the operation that is applied is indeed an integer division,
and that the type of the result is not a float.
This is achieved by using the QUO_ASSIGN operator, instead of the QUO
operator.
This should fix several problems lurking around, and it notably fixes
one of the visible consequences, which is a systematic panic when using
the REPL as a "calculator".
This incidentally also allows us to revert what was done in
5dfc3b86dc since it now turns out it was
just a hack to fix one of the symptoms.
Fixes#864
As per https://golang.org/cmd/go/#hdr-Build_constraints,
using GOOS=android also matches tags and files for GOOS=linux,
so exclude it explicetly to avoid collisions.
Also using GOOS=illumos matches tags and files for GOOS=solaris.
Fixes#843.
This change fixes two data races related to composite literal creation.
The first one isn't controversial as it is just about initializing the
variable that contains the values in the right place, i.e. within the
n.exec, so that this variable is local to each potential goroutine,
instead of being racily shared by all goroutines.
The second one is more worrying, i.e. having to protect the node typ
with a mutex, because a call to func (t *itype) refType actually
modifies the itype itself, which means it is not concurrent safe.
The change seems to work, and does not seem to introduce regression, but
it is still a concern as it probably is a sign that more similar
guarding has to be done in several other places.
This change allows the interpreter to execute tests and benchmarks
functions provided by packages.
The test subcommand is similar to the "go test" command and
all the relevant flags have been kept.
The ability to evaluate a directory or a package has also been added.
A new method Symbol to access exported symbol values of an interpreted
package has been added. This method is used by the test subcommand.
An EvalTest method has been added to evaluate all Go files, including "*_test.go".
The testing packages from the standard library have been added to stdlib used
symbols.
Struct fields of type interface must be converted in wrapper
values to be reachable by the runtime. Call genInterfaceWrapper
on such values.
Fixes#832.
When searching for a binary method on structures, look up on embedded
fields first, otherwise the resulting index is incorrect, as
reflect.Type.MethodByName succeeds also on container struct.
Fixes#834.
This change fixes two distinct data races:
1) some global vars of type *itype of the interp package are actually
mutated during the lifecycle of an Interpreter. Even worse: if more than
one Interpreter instance are created and used at a given time, they are
actually racing each other for these global vars.
Therefore, this change replaces these global vars with generator
functions that create the needed type on the fly.
2) the symbols given as argument of Interpreter.Use were directly copied
as reference (since they're maps) when mapped inside an Interpreter
instance. Since the usual case is to give the symbols from the stdlib
package, it means when the interpreter mutates its own symbols in
fixStdio, it would actually mutate the corresponding global vars of the
stdlib package. Again, this is at least racy as soon as several
instances of an Intepreter are concurrently running.
This change fixes the race by making sure Interpreter.Use actually
copies the symbol values instead of copying the references.
The goal is to provide greater control of input, output and error
streams of the interpreter. It is now possible to specify those
as options when creating a new interpreter. The provided values
are propagated to relevant stdlib symbols (i.e fmt.Print, etc).
Care is taken to not update the global variables os.Stdout, os.Stdin
and os.Stderr, as to not interfere with the host process.
The REPL function is now simplified. The deprecated version is removed.
The tests are updated to take advantage of the simplified access
to the interpreter output and errors.
Fixes#752.
When running TestEvalScanner with -race=true, one can observe a data race such as:
```
WARNING: DATA RACE
Read at 0x0000029f3d68 by goroutine 52:
github.com/containous/yaegi/interp.(*itype).defaultType()
/Users/mpl/src/github.com/containous/yaegi/interp/type.go:1466 +0x572
...
github.com/containous/yaegi/interp.(*Interpreter).EvalWithContext.func1()
/Users/mpl/src/github.com/containous/yaegi/interp/interp.go:501 +0xf0
Previous write at 0x0000029f3d68 by goroutine 43:
github.com/containous/yaegi/interp.(*itype).refType()
/Users/mpl/src/github.com/containous/yaegi/interp/type.go:1419 +0x854
github.com/containous/yaegi/interp.(*itype).TypeOf()
/Users/mpl/src/github.com/containous/yaegi/interp/type.go:1427 +0xa6
...
github.com/containous/yaegi/interp.(*Interpreter).EvalWithContext.func1()
/Users/mpl/src/github.com/containous/yaegi/interp/interp.go:501 +0xf0
```
Before this change, since closing the pipe to the REPL is done in a defer, it
means that all the i.REPL calls (and hence each goroutine for each of these
calls) are kept running and alive until the very end of the test. It should not
matter, since a new interpreter is created for each test case, and thus all the
i.REPL calls should be completely independent from each other.
And yet, by wrapping each test case in a function call, and thus making each
i.REPL call terminate as soon as the test case is over, the data race seems to
be fixed. This could suggest that the separate i.REPL calls from separate
interpreter instances are somehow sharing some memory, but I do not know how to
explain that.
The problem has yet to be fully understood, but at least this change restores
the test, without making the CI fail again.
The Go parser is able to detect some (but not all) (re-)decleration errors,
if the DeclarationErrors flag is enabled, which was not done so far.
This PR therefore enables that flag, which allows the removal of some of
the now unneeded code that was recently added to support redeclarations.
Fixes#811
In interactive mode, a line starting with the "func" keyword is usually
"wrapped", by prepending to it a "package main" statement, to make it
a valid piece of Go source code.
However, when the line is actually an anonymous function call, such as:
func() { println(3) }()
then this wrapping is not enough, as this is not valid Go in the global
context. Therefore, this kind of of expression must also be wrapped
inside a main func (as is the default case for most REPL inputs).
Since the detection and handling of such a case turned out to be quite
unelegant, this PR instead introduces a retrying phase when a parsing
error occurs for a particular class of cases. That is to say, when a
"func expression" wrapped in a main package fails to be parsed, it is
then wrapped in a main func before parsing is retried.
N.B. TestEvalScanner has been disabled for this change, because the additional test cases revealed a (most-likely already existing) data race.
Fixes#721
The input scanning is now performed in a sub goroutine and
the interrupt is listened in another goroutine, either to cancel Eval
or to cancel the current line scan.
Entering a '\n' after a 'Ctrl-C` to get the prompt is
not necessary anymore.
When determining the size of an array and a symbol is found, the symbol must be a const for the type to be valid.
While it makes sense for this check to be done in type checking, the type can be determined by GTA which would then fail. For now this check is done when getting the node type.
Fixes#825
The trick is that in reflect, the field is called Anonymous, which is
actually a different notion in the Go spec/vocable.
n.b. only works for non-recursive types for now.
Fixes#781
The recent changes that added some redeclaration checks implicitly added more
strictness related to namespaces and scopes which, among other things, broke
some uses that "accidentally" used to work.
For example, given
const script1 = `
import "fmt"
// more code
`
const script2 = `
import "fmt"
// some other code
`
If one Evals script1, then script2, with the same interpreter, without
specifying any scope, as the two fragments would be considered part of the same
(.go file) scope by default, a redeclaration error would be triggered because
import "fmt" is seen twice.
A work-around would have been to specify (a different) i.Name before each Eval
call, so that each script is considered as coming from a different .go file, and
hence are respectively in different scopes with respect to imports.
That lead us to realize we had to make specifying things such as file-scope, and
"incremental mode" (aka REPL), more obvious in the context of an Eval call.
In addition, we want to lay down the foundations for Yaegi being able to behave
more like the go tool wrt to various inputs, i.e. it should be able to take a
package directory, or an import path, as input, instead of just a .go file.
Hence the introduction of a new kind of Eval method (whose signature is not fixed yet):
func (interp *Interpreter) EvalPath(path string) (res reflect.Value, err error)
It partially solves the problem described above because:
1. the path given to EvalPath can be used as the file-scope hint mentioned
above, for now (even though the related implementation details might change).
2. Eval always runs in incremental mode, whereas EvalPath always runs in
non-incremental mode, hence clarifying the situation in that respect.
And to avoid confusion, the Name field of Interpreter is now non-exported,
since it is somewhat redundant with the path argument of EvalPath.
Note that #731 is not fully fixed (and might never be), as a requirement of the
proposed solution is to move the input bits of code into respective files
(instead of leaving them as strings).
Finally, some related bugfixes, documention changes, and some refactoring have
been included. Notably, there is no "empty scope" anymore, i.e. name defaults
to "_.go" when it is not specified.
Updates #731Fixes#778Fixes#798Fixes#789
Co-authored-by: Marc Vertes <mvertes@free.fr>
This adds type checking to SliceExpr as well as handling any required constant type conversion.
It should be noted that the check that `high` and `max` are not nil in a 3-index slice has been left out as this check is handled before type checking. Tests have been included for these cases.
In a couple of occurences, tests with enabled race detector exposed
some concurrent accesses to the cancelation callback used in select and
channel operations send and recv for EvalWithContext. This change ensure
that all accesses to this object are protected by mutex.
Fixes#815.
In the case of a pointer or alias composite literal expression, `compositeGenerator` changes the type to remove the pointer or alias. This causes a nested composite literal to have the wrong type.
Instead of changing the node type, the removal of the pointer or alias is moved to the runtime, allowing the node type to remain unchanged. This fixes potential issues in the type checking.
This adds type checking to CallExpr (excluding builtin type checking, as that is a PR in its own right) as well as handling any required constant type conversion.
This also changes constant strings and runes to be represented as `constant.Value`. Runes change `rval` type at CFG typing time to avoid having to type at AST time. There are also changes to importSpecs and `stdlib` to account for the string change. With this all `untyped` types should now be `constant.Value`s, although errors are still not returned if this is not the case to be sure we do not break things.
This also fixed a bug in `itype.methods` that would panic if the type was recursive.
Avoid goroutines leak, accumulation of defered functions and
spurious resets of signal handlers. Effectively catch interrupt
signal (Ctrl-C) to cancel current eval.
Fixes#713.
So far the REPL loop was treating any parsing error coming from
go/parser to generate the AST, as having occurred because the source
code was not yet complete (unfinished block). And it was therefore
ignoring all of them.
However, some of these errors are legitimate, and must be caught as soon
as they occur, otherwise the REPL cycle would stay in an errored state
forever (even when the block terminates), without the user getting any
feedback about it.
Therefore, this change adds an extra check when a parsing error occurs,
i.e. it verifies that it looks like an "EOF" error (unfinished block)
before it ignores it (as the user is supposed to terminate the block
eventually). Otherwise the error is treated just like a "non-parsing"
(cfg, gta, ...) error and printed out.
Fixes#637
This adds type checking to both `IndexExpr` and `CompositeLitExpr` as well as handling any required constant type conversion.
This includes a change to the type propagation to the children of a composite literal. Previously in most cases the composite literal type was propagated to its children. This does not work with type checking as the actual child type is needed.
Fix#787 changes how interfaces are set on a struct (compositeSparce). This change however makes calling the interface panic.
This PR reverts part of the change in #787 and adds a test to ensure it does not break again.
The Yaegi command line has been changed to provide subcommands.
The following sub-commands are provided:
- extract (formerly goexports)
- help
- run
- test
The previous behaviour is now implemented in run command which
is the default, so the change should be transparent.
In run command, prepare the ability to run a package or a directory
in addition to a file. Not implemented yet
The test command is not implemented yet.
The extract command is meant to generate wrappers to non stdlib
packages.
Fixes#639
The detection of special cases of interpreter functions and interfaces
is more precise. It allows at least to complete parsing of
code where type is derived from a type assertion expression.
Fixes#770.
* interp: support another vendoring case
Namely, when the vendor dir is a sibling (or an uncle) relative to the
current pkg
Fixes#758
* make linter happier
* address review comments
* fix, cleanup, add unit tests
* add dummy files to force dirs into git
The range init AST execution was skipped, and range could work
only over variables or direct function calls. By setting the
start node to the start of init and not init itself, we ensure
that the init AST is always taken into account.
Fixes#775.
* fix: constant definition loop on out of order vars
* fix: do not wire global varDecl
* fix: wire and execute global vars
* chore: add tests
* fix: refactor and lint
By returning early in case of incomplete type in CFG, we avoid
a panic, and let a chance to a new attempt after the missing
type has been parsed.
Fixes 763.
* feature: restrict symbols which can exit the interpreter process
Some symbols such as os.Exit or log.Fatal, which make the current process
to exit, are now restricted. They are replaced by a version which panics
instead of exiting, as panics are recovered by Eval.
The restricted os.FindProcess version is identical to the original
except it errors when trying to return the self process, in order to
forbid killing or signaling the interpreter process from script.
The os/exec symbols are available only through unrestricted package.
The original symbols are stored in an unrestricted package, which
requires an explicit Use, as for unsafe and syscall packages.
The Use() interpreter method has been slightly modified to allow inplace
updating of package symbols, allowing to replace some symbols but not
the entire imported package.
A command line option -unrestricted has been added to yaegi CLI to use
the unrestricted symbols.
Fixes#486.
* fix: lint
* fix: switch type from valueT in struct case
In a struct case in type assertion, if the source is a valueT, we still need to take the struct type to allow method and field resolution.
* fix: handle all ptr structs as well
In all situations where the results are set directly
to the frame, and not using a value helper, the right level of
indirections must be applied, otherwise we may end-up writing
in the wrong frame (the local one, instead of a caller or global).
Fixes#735.
* fix: copy the type in recursion
In more advanced recursive cases, setting the rtype to interface may interfear with typeing. To stop this from happening, instead of setting t.val.rtype to interface in the hope it will be set correctly later, a copy if the type is made, and the rtype of the copy is set to interface{}.
* fix: detect intermediate recursive structs
In the case of a nested recussion, each symbol can have
a different perspective on the recursion. In this case,
it is impossible to move from one struct to the next.
To keep the perspectives the same, any intermediate struct
that contains a recursion should also be set to interface{}.
so that all perspectives are the same.
* fix: handle arb recursion
* chore: refactor dref to be consistent
* fix: invalid recursive struct issue
* fix: handle checkptr issue
* fix: move unsafe into function to stop ptr check
* fix: handle deref in assign
When operations write their result to a non-local frame, the node
level field must be set accordingly, otherwise they attempt to write
in the wrong frame.
Fixes#730.
* fix: untyped constant cconverson to default type
In definition assign expression, the source type is propagated to
the assigned value. If the source is an untyped constant, the
destination type must be set to the default type of the constant
definition. A fixType function is provided to perform this.
In addition, the type conversion and check of constants is
refactored for simplifications.
Fixes#727.
* test: fix _test/const14.go
The standard library syscall package for Solaris defines unimplemented
symbols Syscall6 and RawSyscall6 which makes the build fails on
Solaris platform, now that yaegi command imports syscall symbols.
As the standard library package is locked down, this will remain
unchanged. We just skip those symbols.
Fixes#725.
* fix: make interpreter methods discoverable by runtime
When generating an interface wrapper, lookup existing wrappers by method
to get the one with the biggest set of methods implemented by interpreter.
A string method is also added to wrappers, in order to provide a string
representation of the interpreter value rather than the wrapper itself
(at least for %s and %v verbs).
This allows the runtime to pickup an interpreter method automatically
even if the conversion to the interface is not specified in the script. As
in Go spec, it is enough for the type to implement the required methods.
A current limitation is that only single wrappers can be instantiated,
not allowing to compose interfaces.
This limitation can be removed when the Go reflect issue
https://github.com/golang/go/issues/15924 is fixed.
Fixes#435.
* test: add a simpler test
* fix: detect default comm clause in select from AST
The heuristic to distinguish a default comm clause was too weak.
Make it robust by using AST.
Fixes#646.
* rename test to avoid conflict
* fix: perform send channel action for select
The CFG was wrong for select send comm clauses. If an init operation
on channel was required (like a derefence, index operation, ...) it
was skipped. The bug was invisible in case of a local var channel.
A send channel init operation consist to prepare both the data to
send (right subtree of send AST) and the channel itself (left
subtree of send AST). All channel init operation must be performed
prior to call select.
Fixes#647.
* doc: fix comment
* invert test to continue early
please note that this also changes the logic a little bit, since the
line that was:
if pn != nil {
now implictly becomes:
if an != nil && pn != nil {
(which I think is actually more correct).
* explicit chaining of init actions in select
* explicit chaining of init actions in select
Co-authored-by: mpl <mathieu.lonjaret@gmail.com>
* fix: finish support of type assertions which was incomplete
TypeAssert was optimistically returning ok without verifying that
value could be converted to the required interface (in case of
type assert of an interface type), or not checking the type in
all conditions. There is now a working implements method for itype.
Fixes#640.
* style: appease lint
* fix: remove useless code block
* doc: improve comments
* avoid test conflict
* interp: detect local redecleration
Fixes#666
* make make check happy
* fix typo in filenames, add them to the exclusion list
Co-authored-by: Marc Vertes <mvertes@free.fr>
* fix: use interface wrappers to expose interface values to runtime
If a value is assigned to, or returned as, a binary interface,
then use the interface wrapper generator to convert the value
accordingly.
Fixes#630.
* test: rename NewFoo in Foo
* test: rename NewFoo in Foo
* fix: improve branch flow to reduce indentation
* fix: correct iterator in map range on binary values
The map range iterator was not initialized correctly for values
originating from runtime.
Fixes#641.
* test: deterministic output for map28.go
* add repl input bounds check
* use bytes.Equal instead of Compare
* fix: use strings.HasPrefix to perform boundcheck on REPL input
This allows to keep most of the code unchanged and more readable.
Co-authored-by: Marc Vertes <mvertes@free.fr>
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
