* feat: generate go1.21 files
* chore: update CI
* feat: add support for generic symbols in standard library packages
This is necessary to fully support go1.21 and beyond, which now
provide some generic packages such as `cmp`, `maps` or `slices`
in the standard library.
The principle is to embed the generic symbols in source form (as
strings) so they can be instantiated as required during interpretation.
Extract() has been modified to skip the generic types, functions and
constraint interfaces which can't be represented as reflect.Values.
A new stdlib/generic package has been added to provide the corresponding
source files as embedded strings.
The `Use()` function has been changed to pre-parse generic symbols as
doing lazy parsing was causing cyclic dependencies issues at compiling.
This is something we may improve in the future.
A unit test using `cmp` has been added.
For now, there are still some issues with generic stdlib packages
inter-dependencies, for example `slices` importing `cmp`, or when
generic types or function signatures depends on pre-compiled types
in the same package, which we will support shortly.
* fixup
* fixup
* fixup
* fixup
* fixup
* fixup
* fixes for go1.20
* fix previous
* update unsafe2 for go1.21, skip faky tests
In go1.21, the reflect rtype definition has been move to internal/abi.
We follow this change for maintainability, even if there is no layout
change (the go1.20 unsafe2 is compatible with go1.21).
We have isolated a few problematic tests which are failing sometimes
in go1.21, but work in go1.20, and also in go1.22. Those tests are
skipped if in go1.21. A preliminary investigation can not confirm that
something is wrong in yaegi, and the problem disappears with go1.22.
* add new wrapper for go1.21 package testing/slogtest
* add missing wrapper for go/doc/comment
* add support for slices generic package
---------
Co-authored-by: Marc Vertes <mvertes@free.fr>
Unsafe functions such as `unsafe.Alignof`, `unsafe.Offsetof` and `unsafe.Sizeof` can be used for type declarations early on during compile, and as such, must be treated as builtins returning constants at compile time. It is still necessary to explicitely enable unsafe support in yaegi.
The support of `unsafe.Add` has also been added.
Fixes#1544.
Only structures with one embedded field can be marked anonymous, due to golang/go#15924. Also check only that the method is defined, do not verify its complete signature, as the receiver may or not be defined in the arguments of the method.
Fixes#1537.
The result of the expression giving the size of an array may be an `int` instead of `constant.Value` in case of an out of order declaration. Handle both cases.
Fixes#1536.
The logic of patching reflect struct representation has been fixed in presence of function fields and to better handle indirect recursivity (struct recursive through nested struct).
Fixes#1519.
For methods defined on interfaces (vs concrete methods), the resolution of the method is necessarily delayed at the run time and can not be completed at compile time.
The selectorExpr processing has been changed to correctly identify calls on interface methods which were confused as fields rather than methods (due to the fact that in a interface definition, methods are fields of the interface).
Then at runtime, method lookup has been fixed to correctly recurse in nested valueInterface wrappers and to find embedded interface fields in case of struct or pointer to struct.
Finally, remove receiver processing in `call()`.The receiver is already processed at method resolution and in genFunctionWrapper. Removing redundant processing in call fixes handling of variadic method, simplifies the code and makes it faster.
With those fixes, it is now possible to load and run `go.uber.org/zap` in yaegi. In turn, it makes possible for yaegi to run plugins dependent on zap, such as coraza-waf.
Fixes#1515,
Fixes#1172,
Fixes#1275,
Fixes#1485.
When generating a new type, the parameter type was not correctly duplicated in the new AST. This is fixed by making copyNode recursive if needed. The out of order processing of generic types has also been fixed.
Fixes#1488
For a long time, there was a confusion between aliased types and named types (due to my misunderstanding of alias types in Go at that time). The type category `aliasT` has been renamed into `linkedT`, which is correct semantically.
Aliased types are only declared using `typeSpecAssign`, and its processing is now distinct from `typeSpec` statement, used for named types.
A `linkedT` type is obtained by a statement like `type A uint32`, where `A` type category is therefore `linkedT`.
An aliased type is obtained by a statement like `type A = uint32` (notice the `=` sign, translating into `typeSpecAssign`).
The semantic difference is that in the first linkedT form, `A` and `uint32` are 2 distinct types, instead of being strictly equivalent in the `typeSpecAssign` form (the 2 names lead to one type definition).
Fixes#1416.
In case of forward definition of a constant, a symbol may be undefined when attempting to compute the array size in type analysis. Just mark the type as incomplete instead of aborting directly, to allow resolution at a second pass.
Fixes#1470.
Up to now functions could be stored as node values in frame (as for interpreter defined functions) or function values, directly callable by the Go runtime. We now always store functions in the later form, making the processing of functions, anonymous closures and methods simpler and more robust. All functions, once compiled are always directly callable, with no further wrapping necessary.
Fixes#1459.
Those declarations involve the indexListExpr AST token, which was not handled in type.go. The same processing as for a single type parameter is applied.
Fixes#1460.
In type assertion at compile time, compare signatures between function types only.
Make `itype.numOut()` return the correct value for Go builtins (this was not strictly necessary due to above fix, but it is correct and improves maintainability).
Fixes#1454.
The representation of non empty interfaces defined in the interpreter is now identical between refType() and frameType() functions, which are used to generate interpreter objects.
Fixes#1447 and #1426.
Status:
* [x] parsing code with generics
* [x] instantiate generics from concrete types
* [x] automatic type inference
* [x] support of generic recursive types
* [x] support of generic methods
* [x] support of generic receivers in methods
* [x] support of multiple type parameters
* [x] support of generic constraints
* [x] tests (see _test/gen*.go)
Fixes#1363.
The type.val field was always pointing to the final underlying type
for aliased types, defeating a possible match if a method was
attached to a type in between. Now the complete chain of aliases
is always preserved.
We have added an underlying() itype method which returns the underlying
type of a defined type (aliasT), even in the presence of multiple
indirections.
We have added a definedType function which checks if type t1 is
defined from type t2 or t2 defined from t1, required when checking
assignability of aliasT types.
Fixes#1411.
PS: this is the 2nd attempt, as the first version #1412 wasn't passing
_test/issue-1408.go as well. This PR does pass and supersedes #1412.
* interp: fix creation of binary composite types
Use direct assignment instead of reflect.Value Set method to
initialize a binary composite type, as for non binary types.
It ensures that a new reflect.Value is stored in the frame
instead of modifying a possibly existing one, which can defeat
the purpose of initializing variables in a body loop.
While there, remove the need to have and use a mutex on types.
Fixes#1381.
* review: rework a bit the test
Co-authored-by: mpl <mathieu.lonjaret@gmail.com>
* interp: handle struct with multiple recursive fields
In case of a recursive struct with several recursive fields of
different type, only the first one was properly fixed when
constructing the corresponding reflect type. We now memorize and
process all fields at the same depth level.
Fixes#1371.
* Update interp/type.go
Co-authored-by: mpl <mathieu.lonjaret@gmail.com>
* fix lint
* fix comment
Co-authored-by: mpl <mathieu.lonjaret@gmail.com>
This comes from experiments looking into #1259 where incomplete twice seen types are marked as complete. To mitigate the problem instead of a map of seen types in `nodeType` a slice is used as a cheap way to keep track of our current path through the node tree.
The current `nodeType` selector precedence is heavy handed in favour of package type. It seems to often create `typeSym` symbols as variable types in the scope will never be found. To fix this if the ancestor node is a field expression, the package type is searched for. After this, if the type is still `nil` the normal scope is searched using `nodeType2`.
Fixes#1158
Make sure to keep always a single copy of incomplete type structures.
Remove remnants of recursive types processing.
Now `import "go.uber.org/zap"` works again (see #1172), fixing regressions
introduced since #1236.
When `nodeType` recovers names and methods, it can overwrite the data if the type is an aliasT. When aliasing a type, do not recover the methods, this will be done in the GTA typeSpec pass.
Related to #1158
It was initially assumed that `nodeType` needed to rebuild the type from scratch, but this is not the case anymore. The existing type constructors are now used in `nodeType` to make it more readable. In doing this some bugs in type strings were found and fixed, along with adding the real package name to the type.
As `ptrOf` is now the only place that pointer types are constructed, it is feasible to cache the pointer type on the value type. To ensures that we have a consistent pointer type for any value type.
As the unsafe and pointer methods in `reflect` are to be depreciated, and seeing no replacement functions, it is now forced that some unsafe is needed to replace this as when and interface is dereferenced it is made unsettable by reflect.
With this in mind, this adds real recursive types by hot swapping the struct field type on the fly. This removes a lot of compensation code, simplifying all previous cases.
**Note:** While the struct field type is swapped for the real type, the type string is not changed. Due to this, unsafe will recreate the same type.
This adds `itype.str` which is a string representation of the type built when the type is built. The goal is to make type comparison simpler and centralise the creation of types just to constructors and `nodeType`. `nodeType` continues to build types in parts so to reuse underlying types better.
Store the interpreter value of the interface object to wrap as
a field called IValue, at offset 0 in wrapper structures.
Update extract to include IValue field.
In typeAssert, detect interface wrapper, and dereference the
interpreter value from IValue wrapper field.
Fixes#1166.
In parsing array type declaration, The type check of array size was
restricted to `int`. Broaden the test to accept any valid integer
kind.
Fixes#1175.
The resolution method was not compliant with the Go specification which
requires to retain the object where the field or method is the most
shallowed.
The detection of ambiguous fields or methods (same depth in different
objects) has also been added.
Fixes#1163.
In selector resolution, struct field matching now precedes
method matching. Before struct field matching could be skipped
in case of a matching method, which is incorrect, as demontrated
by _test/issue-1156.go.
Field lookup has been fixed to operate on recursive structures.
Concrete type values are derived when filling a receiver for
interface methods.
LookupBinField has been fixed to skip non struct values.
LookupMethod has been fixed to iterate on interface values as
well as concrete type values.
Fixes#1156.
In typecheck.go, detect binary methods so we know when to skip the receiver as first parameter when checking function signatures. The signature check is not yet performed, we just avoid a false error.
In cfg.go, take care to label types with isBinMethod field to true whenever a binary method is resolved.
Also, do not attempt to wrap node in functions if the node value is already a binary function.
Fixes#1145.
