Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[pull] develop from enso-org:develop #92

Open
wants to merge 176 commits into
base: develop
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented Jan 9, 2025

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.1)

Can you help keep this open source service alive? 💖 Please sponsor : )

A wild attempt at fixing MacOS signing process that prevents nightlies from working.
Also cleaned up dependencies for wrapper projects - `scala-compiler` was unnecessary and was introducing jline requirement.
Related to #11957.

# Important Notes
Not sure if this will fix the nightly as I'm still unable to confirm the real issue but this doesn't hurt.
@pull pull bot added the ⤵️ pull label Jan 9, 2025
Akirathan and others added 28 commits January 9, 2025 13:18
… mismatch (#12026)

This is a quick fix for the [IR cache version mismatch problem](#12023 (comment)).

#### Why the benchmark is failing in the first place?
The problem was encountered first in this job: https://github.com/enso-org/enso/actions/runs/12680614808/job/35342759884?pr=12023

I believe these are the steps that happened:
1. Some previous job filled in the global IR caches in `~/.local/share/enso/cache` (with older version) - for example stdlib tests that run something like `enso --run test/Base_Tests`.
2. The failing job did `buildEngineDistribution` which fills in the local IR caches in `built-distribution` with newer version but does not touch the global caches. (https://github.com/enso-org/enso/actions/runs/12680614808/job/35342759884?pr=12023#step:7:2003)
3. `ImportStandardLibraryBenchmark` [was run](https://github.com/enso-org/enso/actions/runs/12680614808/job/35342759884?pr=12023#step:7:2620) and it read the IR caches from the global location, instead from the local location, thus the compiler produced an output and an [`AssertionError` was thrown](https://github.com/enso-org/enso/actions/runs/12680614808/job/35342759884?pr=12023#step:7:2626)
- Fix asset rows no longer being memoized
- Regression *likely* introduced by #11937
- The fix (`[...columns]` in `AssetsTable.tsx`) is unrelated though which is weird

# Important Notes
None
Fixes #11702

Updated dependencies, version-locked ones that caused trouble during update. There are many formatting changes due to prettier version bump.
The overall description is the same as in #11267, but this approach tries to generate super classes as suggested in https://github.com/enso-org/enso/pull/11267/files#r1869342527

# Important Notes
Apart from the annotation processor implementation and tests, [CallArgument.Specified](https://github.com/enso-org/enso/blob/80509a1f2420e97b3879fc8e925a9a863373826d/engine/runtime-parser/src/main/java/org/enso/compiler/core/ir/CallArgument.java) was migrated to the new approach from Scala case class. I uploaded its generated class code in [this gist](https://gist.github.com/Akirathan/0e43de77fe91da399406c1ee7ac2cecd) so you can see it without compilation.
- Respect number order when sorting assets by title
- `New Project 1` < `New Project 2` < `New Project 10` rather than lexicographic sort (`1` < `10` < `2`)
- Use user's locale when sorting names

# Important Notes
None
* Remove native libs of opencv or different platforms

* Fix Mac os name

* Include Mac native libs in signing
Versions 2.5+ are causing a strange error: in dev server, the project view is not loaded at all:

![image](https://github.com/user-attachments/assets/33c1464f-02e5-4af0-9722-c09683e50677)

Nothing is printed to logs. When switching tab back to assets panel, an exception is raised from inside veaury.
Previously, a default MANIFEST file would be used, which would be
problematic for modules.
Previosly we were wrongly relying on the presence of the file. That way,
a bash script meant that NI integration was wrongly used.
This change uses Tika, but it has already been present in other
subprojects so no additional dependency shall be added.

Follow up on #11880.
* Add BuiltinsExposeMethodsTest

* typo in docs

* Test iteartes all the builtins and checks that all methods are invocable member

* Add TypesExposeConstructorsTest

* Naive implementation of BuiltinObject base class.

* Ref extends BuiltinObject

* Builtin type anot processor ensures that a class must extend BuiltinObject

* Enrich BuiltinsExposeMethodsTest

* All builtin types extend BuiltinObject

* Text is BuiltinObject

* EnsoBigInteger is BuiltinObject

* BuiltinObject does no asserts in constructor

* ArrayProxy is BuiltinObject

* Test skips host values and Nothing

* fmt

* Remove outdated test.

No longer true what this test tested

* EqualsComplexNode: Timezone and duration are not treated as object with members

* Fix DebuggingEnsoTest - Date in js is date time, not date

* Fix DateTest - ensoDate now has members

* Add interop.readMember test to BuiltinsExposeMethodsTest

* VectorSortTest is executed in context

* Add tests that invoke builtin methods on particular builtin types

* member methods on BuiltinObject are behind TruffleBoundary

* Cache builtin type in BuiltinObject.

This fixes native image build of engine-runner.

* Reuse context in DebuggingEnsoTest.

This fixes the AssertionError in invalid sharing layer.

* Reuse hardcoded builtinName constants from annotation

* Move BuiltinObject to package org.enso.interpreter.runtime.builtin

* Fix FQN of BuiltinObject in the annotation processor

* Make exported messages on BuiltinObject final

* Update generated class names in Builtins.

Fixes compilation after 8acb04a

* Exported messages in BuiltinObject are not behind TruffleBoundary

* [WIP] Add BuiltinsJavaInteropTest

* Storage.vectorizedOrFallbackBinaryMap accepts Value as parameter

* Builtin types expose methods, not BuiltinObject

* BuiltinObject has no members

* Remove test Text.is_empty

Text is a builtin type with Builtins module scope, which does not have `is_empty` method.

* Fix tests - invocation is done via static methods

* Type.InvokeMember uses InvokeFunctionNode instead of UnresolvedSymbol

* fmt

* Type.InvokeMember prepends receiver argument

* Test invocation of File.path

* Test invocation of Vector.to_text

* Method fetching is done behind truffle boundary

* Reassigning @CompilationFinal field should deoptimize

* Fixes after merge

* Add TruffleBoundary

* Revert DateTest

* Fix PrivateConstructor test

* Methods on Type are internal members

* fmt

* Add TruffleBoundary

* Revert Storage to develop

* Remove FIXME comment

* Simplify usages of Ref.get and Pth.root

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
- Every `Temporary_Directory` should have `cleanup` registered in the `teardown` of a test group: added 2 missing ones and a doc reminding to do that.
- Tried automatically detecting missed ones but simple solution failed to work - not continuing for now.
- Added `Image_Tests` to cloud test run as some data link integration tests are there.
Closes: enso-org/cloud-v2#1630

1. Invalidate remote backend queries when user goes online
2. Moves auth-related logic from `AuthProvider` to `SessionProvider`
3. Do not log user out if `users/me` returns 401
Created a bazel-based build system, implemented a whole GUI build target within it.

# Important Notes
- Running `pnpm install` will install all necessary dependencies, including Bazel itself.
- The `.env` file with safe defaults is now committed to the repository, at `app/gui`. The old version at `app/` is no longer used.
- Because the `.env` file is committed, it does not contain anything sensitive, and developers must create `.env.local` file at `app/gui` and overwrite the variables as needed. Ask @Frizi or @vitvakatu for a template file.
- You might need a few additional things on Windows:

1. `Developer Mode` must be enabled to support creating filesystem symlinks. E.g., using this instruction: https://pureinfotech.com/enable-developer-mode-windows-11/
2. You must create either `.bazelrc.local` at repository root, or `%USERPROFILE%\.bazelrc` (consult https://bazel.build/run/bazelrc for more possible locations), containing the following:

```
# Use different drive letter if needed, but the path must be SHORT
startup --output_base=C:/_bzl
common --disk_cache=C:/_bzl-disk
common --repository_cache=C:/_bzl-repo
```
3. You need to have `bash.exe` available in `PATH`, so either:
- install MSYS2 (https://www.msys2.org/)
- or use bash executable provided with Git by adding the following to your `PATH` variable: `C:\Program Files\Git\bin`, if you have it installed.
- or configure Git installation selecting the third option:
![Git installation settings](https://github.com/user-attachments/assets/def189fa-985b-47f3-8c8b-153c0f39fa26)
* Add failing test

* Error.catch_primitive throws type error on wrong self arg

* Move Error.catch_primitive to a private internal module

* error message contains just single string
- Added all `bazel-*` directories to .gitignore, so any project directory base names are supported.
- Removed usages of `enso-common` from vite configuration script, to avoid the need to run `compile` prior to starting dev server. The in-bundle references are working correctly regardless.
- Fixed version env propagation through IDE rust build script, which hopefully will fix wrong visible version in the nightly build title.
This PR adds new field called `groups` on the User object to avoid fetching the usergroups list to get the names of the groups.

Also this PR removes the `<ManagePermisssionsModal />`
Another steps on the #11462 quest. Putting _warning_ on most of the `types.md` text.
…istribution (#12035)

Introduce `ENSO_LAUNCHER` env var that configures if and how the native image of `engine-runner` is built (More specifically, how `sbt buildEngineDistribution` command behaves):
- `native`: NI is built optimized without assertions and without debugging symbols
- `debugnative`: NI is built not-optimized with assertions enabled and with debugging symbols.
- `shell` (the default value): NI is not built - `buildEngineDistribution` continues to behave as on develop.

Docs in [native-image.md](https://github.com/enso-org/enso/blob/e5fe54105a05fcf3235c57ff722c22a9236528f1/docs/infrastructure/native-image.md#L201-L214)
Allow different suggestion DB types to have different properties, instead of assigning placeholder values; move type-specific entry logic into entry definitions. The `Any` type is now implicit: It is no longer stored as a parent of every type. When filtering by self-type, `Any` is treated as unknown.

Part of #11815.
…12066)

That should allow running IDE with `--engine false`, and still connecting to external local instance of project manager.
Small adjustment to contributing guide.
AdRiley and others added 30 commits February 15, 2025 11:22
* New Test

* New enso class

* New java code

* New java test

* More tests

* Refactor

* Green

* Code review refactor

* fmt

* Fix test

* Code Review Changes
* Enable NI with IDE

If Enso's native image of runner is present, IDE will invoke it.

* Use NI in IDE conditionally

When `ENSO_LAUNCHER=native` NI is being triggered, otherwise the old
behaviour follows.

Added missing resource file.
followup #12264

It fails to clone the repo if you don't have any public SSH keys in your GitHub account.
…eld (#12253)

* Add some tests

* Removed unused variable

* AnyToTextNode does not evaluate suspended field

* Typo in test

* to_text does not evaluate lazy fields

* Already evaluated lazy field is printed

* Implement isFieldEvaluated message in EnsoMultiValue
…orageIterators (#12274)

The first step to get back some of the performance from the storage refactor.
- New `ColumnLongStorageWithArray` and `ColumnDoubleStorageWithArray` interfaces allowing low-level iterators to iterate over the array directly.
- Removed the `DoubleArrayAdapter`, `BigIntegerArrayAdapter` and `BigDecimalArrayAdapter` in favour of having `ColumnStorage` based facades allowing for some storage to present as others.
- Moved the iterations in `NumericBinaryOpImplementation` into `StorageIterators`. Currently a lot of duplication to the `NumericComparison`. Will look to make more common later.

The intention is to keep working on the common iteration layer as build up the operator space.

Various tests on the performance impact of various parts (tested on my PC, over 1,000,000 additions, only provided for reference):
- Using a builder versus populating array and bitset adds about 0.1ms.
- Using the iterator over a tight inline loop appeared to be about 0.2ms
- Using the array loops returns about 0.4ms over the getItemAsLong calls.
Enables more types to exist within the Expression language rather than making everything columns everywhere.

# Important Notes
- The result is still always made into a column. This could be changed in the future but didn't see the need today.
- A Nothing literal in the expression language now means a column of nothings with type nothing (aka a null column)
- There are a couple of places where null columns aren't playing nicely with column operations. I have raised a ticket and commented out the tests.
- I haven't yet done any performance testing. I plan to do that as a follow on and optimise if necesary.
Addresses #12264 (comment)

If someone was surprised that it works (as I was), here is an excerpt from the git docs (emphasis mine):

> <repository> is the URL of the new submodule’s origin repository. This may be either an absolute URL, or (if it begins with ./ or ../), the location relative to the superproject’s default **remote repository**
Trying out to add a log message to help @PabloBuchu resolve a problem with Cloud scheduled workflows.
This PR fixes:

1. Visual regression when icons turn black on an accent button
2. Resets form after closing the Upsert secrets dialog
3. Hides enso devtools by default in prod
* benchmark command accepts optional bench_name

* Add bench_name as workflow input

* lint

* lint
Adds regex support to the expression language in the form of:

- regex(.*)
- r/.*/

e.g.

text_replace([C], regex('l{2}'), '')
or
text_replace([C], r/l{2}/, '')

I need to update the documentation site (help.ensoanalytics.com) in a separate MR

# Important Notes
- I fixed the column name generation for text_replace using a regex which is technically a breaking change. However the old name was so poor I think I am happy to call it a defect.
- I went for r/ / rather than the js style / / mainly for ease of parsing to be able to differentiate from a / divide.
Fixes #12181

The electron package may be built for staging:
```
/run ide build --mode=staging
```
…12329)

First chunk of operations out of the storages.
Creates a new Comparators class which does BiPredicate maps.
Next PR will cover STARTS_WITH, ENDS_WITH, CONTAINS and move the numeric comparators.
- Related to further work discussed in #7359
- Closes #12316
* JavaFinder uses PortableDistributionManager.

So that it detects if it is run from a portable distribution

* Bundle is detected recursively

* log on single line
* limit the size of expressions

* fmt

* clean up

* More cleanup
Fixes #12113

https://github.com/user-attachments/assets/ed2edac9-597b-4b60-b4fd-42b4ca591be4

# Important Notes
Refactored ContextMenu to be more general; also put all actions (previous "buttons") in one module, with plans to merge it with keybinding system as well.
Regression introduced in #12264
I don't know why it wasn't blocked by CI.
* Extract label changed jobs into a separate workflow

* [WIP] Modify Standard.Base API

* Rename jobs

* Not stdlib file changed checking in engine-changed-files.yml

* stdlib API check workflow is generated

* Fix escaping brackets in run step

* ALL_CHANGED_FILES is not escaped

* Generate jobs for the rest of Standard libs

* Revert "[WIP] Modify Standard.Base API"

This reverts commit ccbf82f.

* [WIP] Add API signature of Standard.AWS

* Revert "[WIP] Add API signature of Standard.AWS"

This reverts commit 231f04b.

* Reintroduce StandardLibraryApiCheck
- Moves `STARTS_WITH`, `ENDS_WITH`, `CONTAINS` and `LIKE` out of `StringStorage` and into new design.
- Resolves `MixedStorage` down to the base type in these operations now.
- Removes `StringBooleanOp` and `LikeOp`.
- New standard `BinaryOperation` for the methods.
- Fixed bug with `between` and null value handling.
- Enabled `Null` pending tests.
To speed developer experience up let's use JAR version of project manager when calling
```
sbt:enso> runProjectManagerDistribution
```
unless `ENSO_LAUNCHER=native` is specified or unless `buildProjectManagerDistribution` has explicitly been called.
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.