Clippy Python scripts portablility #2882

mati865 · 2018-06-29T09:23:08Z

Currently all Python scripts in this repository use shebang #!/usr/bin/env python.

The problem

This is bad because python means default system Python which is Python 3 for Arch and scripts don't work.
Other distributions are going to follow it sooner or later because Python 2 won't be maintained since 2020, for an example: Fedora proposal. Also many distributions like Ubuntu, OpenSUSE, Fedora want to ship only with Python 3 preinstalled.

Behaviour on different platforms

I've checked many Linux distributions and all of them had python2, python2.7, python3, python3.X (X differs based on the version).
All I know about OSX is it doesn't have python2 but has python2.7, it would be great if somebody could reply with list of all variants.
MSYS2 (aka windows-gnu) does it just like Linux, no idea about windows-msvc (does #!/usr/bin/env even work there?).

Possible solutions

Use #!/usr/bin/env python2; probably would work everywhere (sometimes it's not installed but available in repo) but note that, Python 2 is almost dead already.
Port scrips to Python 3; I have no idea about OSX and MSVC, should work everywhere else.
RIIR; everything will work on every supported platform, it's win-win but takes more effort.

cc no idea

The text was updated successfully, but these errors were encountered:

llogiq · 2018-06-29T18:53:07Z

Alternative: Port the scripts to Rust.

oli-obk · 2018-06-29T18:56:44Z

RIIR all the things

mati865 · 2018-06-29T21:55:27Z

I feel so dump I haven't thought about it earlier.
Edited OP.

ghost · 2018-07-08T08:02:45Z

How would option 3 work? Would you create a utility binary crate or somehow use a Rust build script?

oli-obk · 2018-07-08T08:48:57Z

Yea, just add a crate and maybe a bat and an sh script for convenient invocation together with a .rustup file pointing to stable so the utils aren't rebuilt whenever nightly is bumped

phansch · 2018-07-11T06:17:26Z

I guess portability is a good reason to use Rust here. Another good thing about using Rust would be that we can add some unit tests to these scripts. The scripts are somewhat complicated already. I remember that the update_lints script had issues before and there are no tests that prevent those issues from appearing again.

phansch · 2018-07-17T21:07:55Z

I would be happy to work on this, actually.

@oli-obk Did you mean the rust-toolchain file instead of .rustup?

oli-obk · 2018-07-18T08:18:08Z

yes

llogiq · 2018-07-18T09:54:50Z

I have looked a bit into this during my ongoing vacation and I believe this could even be written without using regex so a simple .rs file that implements all the functions would work good enough.

phansch · 2018-07-22T07:46:04Z

@llogiq Were you thinking of using Clippys lint registry to do it without Regex? I have a basic version that uses regex now, and it seems simple enough to continue.

llogiq · 2018-07-22T09:11:58Z

No, just str::split and friends. But using regex is fine, too.

Ref: rust-lang#2882

phansch · 2018-10-17T08:34:06Z

Quick summary of what's left to do for update_lints.py:

Handle -c/--check flag, and use it in CI
Generate the pub mod xxx section (gen_mods in update_lints.py)
Generate the module::lint_name section (gen_group in update_lints.py)
Generate the deprecated lints section (gen_deprecated in update_lints.py)

There's also export.py and lintlib.py which need to be rewritten, too.

3388: RIIR update lints: Generate deprecated lints r=flip1995 a=phansch The update script now also generates the 'register_removed' section in `clippy_lints/src/lib.rs`. Also, instead of using `let mut store ...`, I added a new identifier line so that the replacement will continue to work in case `let mut store ...` ever changes. cc #2882 Co-authored-by: Philipp Hansch <dev@phansch.net>

3161: New lint: Unknown clippy lints r=phansch a=flip1995 This is the Clippy version of the `rustc` lint `unknown_lints`. The behavior of this lint is pretty much the same. Before this is merged a small change in the compiler needs to be done: `CheckLintNameResult` needs to be public. See rust-lang/rust#54106 3387: Replace big if/else expression with match r=flip1995 a=mikerite 3388: RIIR update lints: Generate deprecated lints r=phansch a=phansch The update script now also generates the 'register_removed' section in `clippy_lints/src/lib.rs`. Also, instead of using `let mut store ...`, I added a new identifier line so that the replacement will continue to work in case `let mut store ...` ever changes. cc #2882 Co-authored-by: flip1995 <9744647+flip1995@users.noreply.github.com> Co-authored-by: flip1995 <hello@philkrones.com> Co-authored-by: Michael Wright <mikerite@lavabit.com> Co-authored-by: Philipp Hansch <dev@phansch.net>

3388: RIIR update lints: Generate deprecated lints r=phansch a=phansch The update script now also generates the 'register_removed' section in `clippy_lints/src/lib.rs`. Also, instead of using `let mut store ...`, I added a new identifier line so that the replacement will continue to work in case `let mut store ...` ever changes. cc #2882 Co-authored-by: Philipp Hansch <dev@phansch.net>

3399: RIIR update lints: Generate modules section and lint group sections r=flip1995 a=phansch This adds the last missing parts of the generating code. cc #2882 Co-authored-by: Philipp Hansch <dev@phansch.net>

3408: RIIR update lints: Add check mode (update_lints.py rewrite complete) r=oli-obk a=phansch This finishes up the rewrite of `update_lints.py` in Rust. More specifically, this * adds the `--check` flag and handling to clippy_dev * tracks file changes over the different calls to `replace_region_in_file` * only writes changes to files if the `--check` flag is *not* used * runs `./util/dev update_lints --check` on CI instead of the old script * replaces usage of the `update_lints.py` script with an error `./util/dev update_lints` behaves 99% the same as the python script. The only difference that I'm aware of is an ordering change to `clippy_lints/src/lib.rs` because underscores seem to be sorted differently in Rust and in Python. :checkered_flag: cc #2882 Co-authored-by: Philipp Hansch <dev@phansch.net> Co-authored-by: Philipp Krones <hello@philkrones.com>

mati865 · 2020-10-16T07:53:23Z

I think this is done.

ThibsG · 2020-10-16T11:46:37Z

Actually I am not sure it is totally done, unless these two rewriting have been dropped or done by other means?

There's also export.py and lintlib.py which need to be rewritten, too.

mati865 · 2020-10-16T17:42:25Z

They seem to be used only on the CI so there is no portability issue for the developers.

llogiq · 2020-10-16T18:58:45Z

True, we don't have a clippy_dev subcommand to create the lints.json yet, right? lintlib shouldn't be needed anymore AFAIR.

oli-obk added the E-help-wanted Call for participation: Help is requested to fix this issue. label Jun 29, 2018

This was referenced Jul 25, 2018

s/wiki/lint list/ #2958

Merged

update_lints rewrite: Add structure and --print-only #2985

Merged

auscompgeek added a commit to auscompgeek/rust-clippy that referenced this issue Aug 9, 2018

lintlib: Use Python 3 compatible print

9bb68b8

Ref: rust-lang#2882

auscompgeek mentioned this issue Aug 9, 2018

lintlib: Use Python 3 compatible print #3019

Merged

phansch mentioned this issue Oct 5, 2018

WIP: update_lints.py RIIR #3266

Closed

This was referenced Oct 16, 2018

RIIR update_lints: Replace lint count in README.md #3325

Merged

RIIR update_lints: use WalkDir instead of read_dir #3320

Merged

RIIR update_lints: Update changelog links #3327

Merged

phansch mentioned this issue Oct 31, 2018

RIIR update lints: Generate deprecated lints #3388

Merged

phansch mentioned this issue Nov 2, 2018

RIIR update lints: Generate modules section and lint group sections #3399

Merged

bors bot added a commit that referenced this issue Nov 3, 2018

Merge #3399

a07c271

3399: RIIR update lints: Generate modules section and lint group sections r=flip1995 a=phansch This adds the last missing parts of the generating code. cc #2882 Co-authored-by: Philipp Hansch <dev@phansch.net>

phansch mentioned this issue Nov 4, 2018

RIIR update lints: Add check mode (update_lints.py rewrite complete) #3408

Merged

phansch added the O-windows label Dec 2, 2018

mati865 closed this as completed Oct 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clippy Python scripts portablility #2882

Clippy Python scripts portablility #2882

mati865 commented Jun 29, 2018 •

edited

Loading

llogiq commented Jun 29, 2018

oli-obk commented Jun 29, 2018

mati865 commented Jun 29, 2018 •

edited

Loading

ghost commented Jul 8, 2018

oli-obk commented Jul 8, 2018

phansch commented Jul 11, 2018 •

edited

Loading

phansch commented Jul 17, 2018

oli-obk commented Jul 18, 2018

llogiq commented Jul 18, 2018

phansch commented Jul 22, 2018

llogiq commented Jul 22, 2018

phansch commented Oct 17, 2018 •

edited

Loading

mati865 commented Oct 16, 2020

ThibsG commented Oct 16, 2020

mati865 commented Oct 16, 2020

llogiq commented Oct 16, 2020

Clippy Python scripts portablility #2882

Clippy Python scripts portablility #2882

Comments

mati865 commented Jun 29, 2018 • edited Loading

The problem

Behaviour on different platforms

Possible solutions

llogiq commented Jun 29, 2018

oli-obk commented Jun 29, 2018

mati865 commented Jun 29, 2018 • edited Loading

ghost commented Jul 8, 2018

oli-obk commented Jul 8, 2018

phansch commented Jul 11, 2018 • edited Loading

phansch commented Jul 17, 2018

oli-obk commented Jul 18, 2018

llogiq commented Jul 18, 2018

phansch commented Jul 22, 2018

llogiq commented Jul 22, 2018

phansch commented Oct 17, 2018 • edited Loading

mati865 commented Oct 16, 2020

ThibsG commented Oct 16, 2020

mati865 commented Oct 16, 2020

llogiq commented Oct 16, 2020

mati865 commented Jun 29, 2018 •

edited

Loading

mati865 commented Jun 29, 2018 •

edited

Loading

phansch commented Jul 11, 2018 •

edited

Loading

phansch commented Oct 17, 2018 •

edited

Loading