git.Repo object leaves open files

[radujipa]

I've made a small tool to help me manage tens of repos in one go, e.g. status, checkout, and pull. I'm having an issue when printing details of the last commit.

When I also call the following function, the traceback starts happening:

def get_last_commit_details(repo):
    """Get the HEAD details for a given repository (similar to git logger).

    Args:
        repo (git.Repo): The repository to get data from

    Returns
        str: With details of the last commit in the repository
    """
    details = ''

    # Catches a bug in gitpython lib which leaves opened files...
    try:
        details += '\t' + colourise(
            'commit {sha}'.format(sha=repo.active_branch.commit.hexsha),
            Colour.COMMIT
        ) + '\n'
        details += '\tAuthor: {name} <{email}>'.format(
            name=repo.active_branch.commit.author.name,
            email=repo.active_branch.commit.author.email
        ) + '\n'
        details += '\tDate: {datetime}'.format(
            datetime=repo.active_branch.commit.authored_datetime
        ) + '\n'
        details += '\t' + repo.active_branch.commit.message.strip().split('\n', 1)[0]
    except:
        import traceback
        logger.error(traceback.format_exc())
    return details

Example output:

... bunch of other repos ...
info repo1 is on branch master and is clean
info    commit 599dba58194230af66cc5ea5300981458ded5430
info    Author: Person1 <email@email>
info    Date: 2017-06-02 17:02:10+01:00
info    Share: Emit signals when properties are initialised
info repo2 is on branch master and is clean
info    commit 11218a40c71c0386c4449a87546c1c118235e24a
info    Author: Person1 <email@email>
info    Date: 2017-05-09 15:40:27+01:00
info    Notifications: Add mechanism to update a notification
info repo3 is on branch master and is clean
error Traceback (most recent call last):
error   File "dev-team/dev_scripts/repo-mang", line 289, in get_last_commit_details
error     ) + '\n'
error   File "/Library/Python/2.7/site-packages/git/refs/symbolic.py", line 200, in _get_commit
error     obj = self._get_object()
error   File "/Library/Python/2.7/site-packages/git/refs/symbolic.py", line 193, in _get_object
error     return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
error   File "/Library/Python/2.7/site-packages/git/objects/base.py", line 64, in new_from_sha
error     oinfo = repo.odb.info(sha1)
error   File "/Library/Python/2.7/site-packages/git/db.py", line 37, in info
error     hexsha, typename, size = self._git.get_object_header(bin_to_hex(sha))
error   File "/Library/Python/2.7/site-packages/git/cmd.py", line 1072, in get_object_header
error     cmd = self._get_persistent_cmd("cat_file_header", "cat_file", batch_check=True)
error   File "/Library/Python/2.7/site-packages/git/cmd.py", line 1055, in _get_persistent_cmd
error     cmd = self._call_process(cmd_name, *args, **options)
error   File "/Library/Python/2.7/site-packages/git/cmd.py", line 1010, in _call_process
error     return self.execute(call, **exec_kwargs)
error   File "/Library/Python/2.7/site-packages/git/cmd.py", line 735, in execute
error     raise GitCommandNotFound(command, err)
error GitCommandNotFound: Cmd('git') not found due to: OSError('[Errno 24] Too many open files')
error   cmdline: git cat-file --batch-check
... bunch of other repos ...

My platform details

OS: macOS High Sierra 10.13.2

$ python -c "import git; print git.__version__"
2.1.8

$ python --version
Python 2.7.10

$ git --version
git version 2.15.0

I've tried a mixture of calling del on the repo objects, looking for explicit close methods through the library API - no luck.

Any ideas?

[radujipa]

I managed to avoid tracebacks by calling repo.__del__() after handing each repo.

[Byron]

Thanks for the detailed information, as well as a workaround.
I remember that there was a lot of time spent working around not having deterministic destruction of objects, without ever getting to a turn-key solution that just works.
I believe it's something we have to live with, and I hope this ticket will help others to solve similar problems.

[ankostis]

@radujipa maybe you want to look at tips and suggestions in the various tag.leaks discussions:
https://github.com/gitpython-developers/GitPython/issues?utf8=%E2%9C%93&q=label%3Atag.leaks+