Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Remove duplicated packages #1014

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pre_commit_hooks/requirements_txt_fixer.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,11 @@ def __lt__(self, requirement: Requirement) -> bool:
elif requirement.value == b'\n':
return False
else:
# if 2 requirements have the same name, the one with comments
# needs to go first (so that when removing duplicates, the one
# with comments is kept)
if self.name == requirement.name:
return bool(self.comments) > bool(requirement.comments)
return self.name < requirement.name

def is_complete(self) -> bool:
Expand Down Expand Up @@ -113,10 +118,14 @@ def fix_requirements(f: IO[bytes]) -> int:
if req.value != b'pkg-resources==0.0.0\n'
]

# sort the requirements and remove duplicates
prev = None
for requirement in sorted(requirements):
after.extend(requirement.comments)
assert requirement.value, requirement.value
after.append(requirement.value)
if prev is None or requirement.value != prev.value:
after.append(requirement.value)
prev = requirement
after.extend(rest)

after_string = b''.join(after)
Expand Down
6 changes: 6 additions & 0 deletions tests/requirements_txt_fixer_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,12 @@
b'f<=2\n'
b'g<2\n',
),
(b'a==1\nb==1\na==1\n', FAIL, b'a==1\nb==1\n'),
(
b'a==1\nb==1\n#comment about a\na==1\n',
FAIL,
b'#comment about a\na==1\nb==1\n',
),
(b'ocflib\nDjango\nPyMySQL\n', FAIL, b'Django\nocflib\nPyMySQL\n'),
(
b'-e git+ssh://git_url@tag#egg=ocflib\nDjango\nPyMySQL\n',
Expand Down
Loading