Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Conceptual fix for issue #3589 #126

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

brechtm
Copy link

@brechtm brechtm commented Jan 21, 2021

This makes just enough changes to the code to make python-poetry/poetry#3589 work. This is mostly intended to start discussion and collect feedback.

There are (at least) two outstanding issues:

  1. with a from = "src" packages setup, this adds the files in src twice. The resulting tar.gz does seem to be fine, however.
  2. Performance is abysmal when using include = [{ path = "**/*", format = "sdist" }] in a sizeable project. I believe this is because poetry checks for excluded files on a file-per-file bases and doesn't take shortcuts when entire directories are excluded.

Summary of commits:

  • Log all files being added
  • Pass unresolved paths to is_excluded()
  • 'exclude' should be able to override 'include' matches
  • Some remarks
  • Exclude .git directories and files in them by default

Partly resolves: python-poetry/poetry#3589

  • Added tests for changed code.
  • Updated documentation for changed code.

This makes is_excluded work with symlinks.
Note that this behavior was different for files in dirs; see
"if file.is_dir():" block a bit higher in the code.

Related: python-poetry/poetry#1336
Note that this makes a list of all files in .git directories and is
thus very slow.
if self.is_excluded(
include_file.relative_to_project_root()
) and isinstance(include, PackageInclude):
if self.is_excluded(file.relative_to(source_root)):
Copy link
Author

@brechtm brechtm Jan 21, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes include/exclude behavior, breaking three tests. Currently, include always trumps exclude (see also python-poetry/poetry#1336 and python-poetry/poetry#1750), which is too limiting for some use cases.

One solution, without introducing order-dependent include/exclude rules as in a a MANIFEST.in, is to give globbed paths a lower priority than fully-qualified paths. I feel this offers more flexibility while being intuitive at the same time. This would work for the examples given in python-poetry/poetry#1336, such as:

exclude = [
    "assets/sample/**/*"
]
include = [
    "assets/sample/subdir/one.yaml",
    "assets/sample/subdir/two.yaml"
]

But the following would now also work:

include = [
    "**/*"
]
exclude = [
    ".travis.yml"
]

It might be worth looking into assigning different priorities to glob patterns based on how specific they are.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

poetry build: crash on encountering a symlink
1 participant