Disable broken and outdated CI #39467
Conversation
|
Documentation preview for this PR (built with commit 1364c41; changes) is ready! 🎉 |
|
There's a risk of we just forget to reenable it. Is there a good way to do this dynamically (i.e. check which tests the last release fail and skip these)? |
I'll open an issue where we can track which systems are failing, so we don't forget it. The issue tracker might actually be a more visible place in the first place as most people will not check the CI runs.
Not that I know. |
|
Great, thanks for taking care of this. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
Removing things in CI may break other things elsewhere. Especially the developer guide. In particular, https://doc-release--sagemath.netlify.app/html/en/developer/portability_testing. Did you check? The docker images created by CI linux are used by developers. The sage binder repo (https://github.com/sagemath/sage-binder-env) is using them (in particular the "minimal" image). The interactive sage doc relies on them. This PR will break the doc. Changes affecting developers widely should be discussed or at least posted on sage-devel. This PR falls in the category.
We do not remove code for temporary change. Do not invent a new protocol that code is removed to be disabled. Improving CI is good, but the present PR is also destructive. I object to this PR. |
kwankyu
added
disputed
|
@kwankyu What exactly is broken by this PR? |
This comment was marked as abuse.
This comment was marked as abuse.
dimpase
added
s: needs info
and removed
disputed
|
Technically you're right in that this merely remove part of CI that has been broken by some other change made months ago (thus doesn't "in addition" break anything), and reverting this is as simple as a I don't personally use the minimal configurations and have no strong opinion on whether they're actually useful to developers. Of course having more configurations lead to additional maintenance burden and spent computing power (such as the maintenance cost needed now to make the minimal configuration build again), the cost may not be justified by the usefulness to developers/users. Anyway, since the intention of this PR is to temporarily disable the run until the bug is fixed, can't we just comment it out? I mean, we're all volunteers, if anyone volunteers to fix the minimal configurations then of course that will supersede this pull request and everyone are happy [1]… but if nobody does, the feature unfortunately will remain broken. [1] maybe except those who prefer the minimal configurations to be deleted anyway, but I think the usual workflow is deleting a feature requires 1-year deprecation period and/or announcement of some sort. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Did you see the section of the developer guide that I pointed out? The removed chunks of ci-linux are responsible to push various docker images. One of the image, ubuntu-jammy-standard, is used in the CI check workflows for PRs. This PR will collapse the whole CI checks. Other images, ubuntu-jammy-minimal-with-targets and ubuntu-jammy-minimal-with-system-packages, are used in sage-binder-env. This PR will collapse it and the interactive sage doc that is using it. |
|
Can we not escalate this, thanks. Anyway, I think a good compromise is
Do you think this is reasonable? (anyway I'm not a maintainer and would be happy with anything) p/s. @kwankyu , I don't really completely understand what's going on, but if the particular CI has been failing for months, how come the incremental things etc. still work (this is |
Because these two things are tied up together. The main objection I heard is that without the current sage the distro setting up CI runs is more complicated. |
In the present shape it's not very useful, indeed, and it improving it is important. |
How? The CI Linux (I mean the code in .ci and .github) does not change whether gcc is in "_prereq" or not, or whether "standard packages" consists of 500 or 100 packages. Off-topic:
Testing less sage-the-distribution would work, if at all, toward "sooner". |
| max_parallel: 50 | ||
|
|
||
| standard-constraints_pkgs-norequirements: | ||
| if: ${{ success() || failure() }} |
There was a problem hiding this comment.
Just turn this off. Please do not remove things broken. They may be fixed later.
|
|
||
| standard-sitepackages: | ||
| if: ${{ success() || failure() }} |
There was a problem hiding this comment.
Just turn this off. Please do not remove things broken. They may be fixed later.
| max_parallel: 24 | ||
|
|
||
| maximal-pre: | ||
| if: ${{ success() || failure() }} |
There was a problem hiding this comment.
This is intended to test "optional packages" installed as system package.
| docker_push_repository: ghcr.io/${{ github.repository }}/ | ||
|
|
||
| optional: | ||
| if: ${{ success() || failure() }} |
There was a problem hiding this comment.
This is intended to test "optional sage packages"
| targets_optional: '$(echo $(export PATH=build/bin:$PATH && sage-package list :optional: --has-file "spkg-install.in|spkg-install|requirements.txt" --no-file "huge|has_nonfree_dependencies" | grep -v sagemath_doc | grep -v ^_))' | ||
|
|
||
| experimental: | ||
| if: ${{ success() || failure() }} |
There was a problem hiding this comment.
This is intended to test "experimental sage packages"
#39009 puts CI Linux in good shape. |
On one hand, while I don't understand what exactly the work consist of, I'm sure many of us appreciate your effort in creating the releases. On the other hand, I feel you aren't expressing yourself well, which makes it sound like you're "fighting a religious war against anything that is called minimal", which certainly doesn't help your case. So: can you explain how exactly does this pull request make your (@dimpase 's) life easier compared to (#39009 with gcc python gfortran added to prereq and a few more |
as soon as |
|
meanwhile, @kwankyu engages in hooliganism - coming to my random PR (related to macOS, ironically) and leaving a (totally irrelevant there) comment #39492 (comment) Is he looking to get blocked? |
What do you mean by " (1) _prereq contains gcc, python, gfortran (2) _prereq contains gcc, python, gfortran, and most of standard packages ? Unless all standard packages are required to be installed as system packages, "minimal" configuration in CI still makes sense. |
That's a good goal. But there's a technical issue I don't understand — you can remove support for building from source for common packages like gcc python etc. (which if I understood correctly would count toward "what goes into _prereq"), but even if you move everything installable to Can you explain that? Or, more simply: what simplification are you envisioning for the build system? (personally I use conda+meson, so I haven't need to touch the sage-the-distro build system for a long time. So I might make some wrong assumptions here) |
at least these, and the rest of the "toolchain", e.g. patch.
A lot of standard packages are Python packages (numerically, a vast majority). The complexity should be reduced from two sides - putting more packages in _prereq, and getting rid of dependencies of dependencies. |
|
No one would object to reducing complexity of the build system. The build system changes first, and then CI will follow. Not the other way around. As far as I understand, the complexity of the current build system is mainly rooted in the long-held doctrine that sage should be buildable from source without internet connection. Abolishing the doctrine seems the first thing to do to proceed in the direction of your vision. I am all for it, but I think there are people against it... |
you cannot CI with build targets/configs which are not there any more. So it can't be done sequentially.
That's not a problem, just populate your pip cache with all the packages not in upstream/ that Sage needs, and then you don't need an internet connection. |
|
Follow-up is at #39603 for anyone interested in fixing the most serious issues. |
If you makes changes to targets/configs in But if you make changes in the build system without adjusting CI, and thus CI is broken temporarily until someone fixes CI, then that is not ideal. But that is really not a big deal.
OK. |
<!-- ^ Please provide a concise and informative title. --> <!-- ^ Don't put issue numbers in the title, do this in the PR description below. --> <!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method to calculate 1 + 2". --> <!-- v Describe your changes below in detail. --> <!-- v Why is this change required? What problem does it solve? --> <!-- v If this PR resolves an open issue, please link to it here. For example, "Fixes sagemath#12345". --> Many of the CI runs after a new release are failing, for months now, see eg https://github.com/sagemath/sage/actions/runs/12979684199/job/3621812 6145. Some of these failures are genuine (eg a certain package cannot be built on a certain system) and some others are due to constraints of the build system (eg running out of free space). Since there is very little point in senselessly burning energy, all runs that were failing for the last releases are disabled. Once the underlying issues are fixed, they can be easily be re-enabled. Moreover, the "minimal" runs where only a couple of systems packages are installed and most are build using sage are removed, keeping only the "standard" where all available system packages (corresponding to standard sage packages) are installed. New test run: https://github.com/tobiasdiez/sage/actions/runs/1319937223 2/job/36847711005 ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> - [ ] The title is concise and informative. - [ ] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies - sagemath#39251 (to resolve merge conflicts) <!-- List all open PRs that this PR logically depends on. For example, --> <!-- - sagemath#12345: short description why this is a dependency --> <!-- - sagemath#34567: ... --> URL: sagemath#39467 Reported by: Tobias Diez Reviewer(s): Dima Pasechnik, Kwankyu Lee, Tobias Diez, user202729
Many of the CI runs after a new release are failing, for months now, see eg https://github.com/sagemath/sage/actions/runs/12979684199/job/36218126145. Some of these failures are genuine (eg a certain package cannot be built on a certain system) and some others are due to constraints of the build system (eg running out of free space). Since there is very little point in senselessly burning energy, all runs that were failing for the last releases are disabled. Once the underlying issues are fixed, they can be easily be re-enabled.
Moreover, the "minimal" runs where only a couple of systems packages are installed and most are build using sage are removed, keeping only the "standard" where all available system packages (corresponding to standard sage packages) are installed.
New test run: https://github.com/tobiasdiez/sage/actions/runs/13199372232/job/36847711005
📝 Checklist
⌛ Dependencies