Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: Introduce ecosystem tests for popular plugins #127

Merged
merged 27 commits into from
Mar 26, 2025

Conversation

JoshuaKGoldberg
Copy link
Contributor

Summary

Adding an CI job to the eslint/eslint repo that checks changes against a small selection of third-party plugins.

Related Issues

eslint/eslint#19139

@fasttime fasttime added the Initial Commenting This RFC is in the initial feedback stage label Nov 26, 2024
@voxpelli
Copy link

That none of the eslint-community plugins are included feels a bit odd? Since it's somewhat operated under the same umbrella as ESLint itself?

At least one of eg eslint-plugin-n, eslint-plugin-promise or eslint-plugin-security would be good to include?

I also note that the selection criteria is similar to those outlined in the suggested eslint-community governance, and which have gotten feedback / objections / concerns there: eslint-community/governance#1 (comment)

Copy link
Member

@nzakas nzakas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My apologies, I started a review last week, left a bunch of comments, and then forgot to hit "Submit Review".

I think this outlines a good infrastructure, just looking for some clarifications.

Copy link
Member

@aladdin-add aladdin-add left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me! This is an excellent starting point. We can proceed to run it and see how it performs. 🚀🚀🚀

nzakas
nzakas previously approved these changes Feb 6, 2025
Copy link
Member

@nzakas nzakas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the direction of this makes sense now. Nice work.

Co-authored-by: Milos Djermanovic <milos.djermanovic@gmail.com>
The new CI job will, for each plugin:

1. Clone the plugin into a directory named `test/ecosystem/${plugin}`
2. Run the plugin's package installation and build commands with [ni](https://github.com/antfu-collective/ni)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will ni be installed? And will ni automatically install npm (or another package manager) if it's not found locally, and does this also work in a CI environment?

Copy link
Contributor Author

@JoshuaKGoldberg JoshuaKGoldberg Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

installed

npm i -g ni,

automatically install

The nci command will.

I'll make those two points explicit.

Aside: I filed antfu-collective/ni#263 as a docs issue on ni to support this.

2. Run the plugin's package installation and build commands with [ni](https://github.com/antfu-collective/ni)
3. Link the plugin to the current eslint installation
- This will have to be done manually, as ni does not support linking ([ni#85](https://github.com/antfu-collective/ni/issues/85 "ni issue 85: Maybe xxx link can join ni project"))
4. Run the plugin's `test:eslint-compat` script with [ni](https://github.com/antfu-collective/ni)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does ni also install Node.js in the correct version for the package?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but last I checked, these packages are all fine with running on approximately the same Node.js versions as ESLint's CI. As in, they might specify preferred packageManager / .nvmrc versions, but they should pass CI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think we should be fine using actions/setup-node@v4 with Node.js lts/* like it's done in other jobs.

      - uses: actions/setup-node@v4
        with:
          node-version: "lts/*"

(e.g. https://github.com/eslint/eslint/blob/c99db89141f1601abe6f9d398a4b6c126e3a0bdb/.github/workflows/ci.yml#L17-L19).

This will also make sure that npm is installed so we can run npm i -g ni.


See [Plugin Selection](#plugin-selection) below for specifics on which plugins will be included.

> ⚠️ Plugins are currently being asked for feedback on the `test:eslint-compat` script.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's good. If there are any questions I think we can also discuss them in this RFC.

@mdjermanovic
Copy link
Member

@JoshuaKGoldberg just a reminder there are a few questions left.

@JoshuaKGoldberg
Copy link
Contributor Author

👍 I'd been hoping for an answer on the Discord messaging, #127 (comment). I just went ahead now and pushed it as part of the RFC for review.

Copy link
Member

@nzakas nzakas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving.

@nzakas
Copy link
Member

nzakas commented Mar 18, 2025

@mdjermanovic @fasttime any other comments on this?

I feel like there's enough here now to move forward with a PoC.

Copy link
Member

@fasttime fasttime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@nzakas nzakas added Final Commenting This RFC is in the final week of commenting and removed Initial Commenting This RFC is in the initial feedback stage labels Mar 19, 2025
@nzakas
Copy link
Member

nzakas commented Mar 19, 2025

Moving to Final Commenting based on two TSC approvals

Copy link
Member

@mdjermanovic mdjermanovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

@nzakas
Copy link
Member

nzakas commented Mar 26, 2025

Final commenting period has ended, merging.

@nzakas nzakas merged commit d88b194 into eslint:main Mar 26, 2025
5 checks passed
@JoshuaKGoldberg JoshuaKGoldberg deleted the repo-ecosystem-plugin-tests branch March 27, 2025 12:13
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
feature Final Commenting This RFC is in the final week of commenting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants