Poetry-Built Wheel Timestamps Pinned to 1980 and 2016

[mattsta]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Tested on macos and Linux both with a pyenv Python 3.9.1
• Poetry version: pip package Poetry version 1.1.4
• Link of a Gist with the contents of your pyproject.toml file: any new empty project

Issue

Hi Poets,

I don't think I'm going crazy this time, but poetry is building packages with dates pinned to 1980 and 2016, which is eerily similar to the zip format original date limits.

Reproducible on macos and Linux:

matt@webby:/tmp/q/dist$ poetry new nothing
Created package nothing in nothing
matt@webby:/tmp/q/dist$ cd nothing
matt@webby:/tmp/q/dist/nothing$ poetry build
Creating virtualenv nothing-NUlh54qg-py3.6 in /home/matt/.cache/pypoetry/virtualenvs
Building nothing (0.1.0)
  - Building sdist
  - Built nothing-0.1.0.tar.gz
  - Building wheel
  - Built nothing-0.1.0-py3-none-any.whl

matt@webby:/tmp/q/dist/nothing$ zipinfo dist/nothing-0.1.0-py3-none-any.whl
Archive:  dist/nothing-0.1.0-py3-none-any.whl
Zip file size: 1021 bytes, number of entries: 4
-rw-r--r--  2.0 unx       22 b- defN 80-Jan-01 00:00 nothing/__init__.py
?rw-r--r--  2.0 unx       83 b- defN 16-Jan-01 00:00 nothing-0.1.0.dist-info/WHEEL
?rw-r--r--  2.0 unx      390 b- defN 16-Jan-01 00:00 nothing-0.1.0.dist-info/METADATA
?rw-r--r--  2.0 unx      279 b- defN 16-Jan-01 00:00 nothing-0.1.0.dist-info/RECORD
4 files, 774 bytes uncompressed, 475 bytes compressed:  38.6%

And it extracts to the old dates as well:

matt@webby:/tmp/q/dist/nothing/dist$ unzip nothing-0.1.0-py3-none-any.whl
Archive:  nothing-0.1.0-py3-none-any.whl
  inflating: nothing/__init__.py     
  inflating: nothing-0.1.0.dist-info/WHEEL  
  inflating: nothing-0.1.0.dist-info/METADATA  
  inflating: nothing-0.1.0.dist-info/RECORD  

matt@webby:/tmp/q/dist/nothing/dist$ ls -latrh nothing
total 12K
-rw-r--r-- 1 matt matt   22 Jan  1  1980 __init__.py
drwxrwxr-x 5 matt matt 4.0K Jan  2 16:13 ..
drwxrwxr-x 2 matt matt 4.0K Jan  2 16:13 .
matt@webby:/tmp/q/dist/nothing/dist$ ls -latrh nothing-0.1.0.dist-info/
total 20K
-rw-r--r-- 1 matt matt   83 Jan  1  2016 WHEEL
-rw-r--r-- 1 matt matt  279 Jan  1  2016 RECORD
-rw-r--r-- 1 matt matt  390 Jan  1  2016 METADATA
drwxrwxr-x 5 matt matt 4.0K Jan  2 16:13 ..
drwxrwxr-x 2 matt matt 4.0K Jan  2 16:13 .

The tar file has the proper dates:

matt@webby:/tmp/q/dist/nothing/dist$ ls
nothing-0.1.0-py3-none-any.whl  nothing-0.1.0.tar.gz
matt@webby:/tmp/q/dist/nothing/dist$ tar xfvzp nothing-0.1.0.tar.gz 
nothing-0.1.0/nothing/__init__.py
nothing-0.1.0/pyproject.toml
nothing-0.1.0/setup.py
nothing-0.1.0/PKG-INFO
matt@webby:/tmp/q/dist/nothing/dist$ ls -latRh nothing-0.1.0/
nothing-0.1.0/:
total 24K
drwxrwxr-x 5 matt matt 4.0K Jan  2 16:13 ..
drwxrwxr-x 3 matt matt 4.0K Jan  2 16:07 .
drwxrwxr-x 2 matt matt 4.0K Jan  2 16:07 nothing
-rw-r--r-- 1 matt matt  390 Jan  2 16:04 PKG-INFO
-rw-r--r-- 1 matt matt  492 Jan  2 16:04 setup.py
-rw-r--r-- 1 matt matt  297 Jan  2 16:04 pyproject.toml

nothing-0.1.0/nothing:
total 12K
drwxrwxr-x 2 matt matt 4.0K Jan  2 16:07 .
drwxrwxr-x 3 matt matt 4.0K Jan  2 16:07 ..
-rw-r--r-- 1 matt matt   22 Jan  2 16:04 __init__.py

I haven't seen where poetry is actually building the packages (doesn't look like zipfile.ZipFile is used anywhere for creating packages), but something in the source-to-dist pipeline is borked.

Also, even looking through my pip cache, I can see projects built with poetry because they have weird timestamps:

Archive:  ./28/2d/6b/560369248c33fc2d882500262bb2b6d3e5d672575f3501fa6f/pytokenizations-0.7.2-py3-none-any.whl
Zip file size: 1791 bytes, number of entries: 6
-rw-r--r--  2.0 unx      475 b- defN 80-Jan-01 00:00 tokenizations/__init__.py
-rw-r--r--  2.0 unx      354 b- defN 80-Jan-01 00:00 tokenizations/__init__.pyi
-rw-r--r--  2.0 unx        0 b- defN 80-Jan-01 00:00 tokenizations/py.typed
?rw-r--r--  2.0 unx       83 b- defN 16-Jan-01 00:00 pytokenizations-0.7.2.dist-info/WHEEL
?rw-r--r--  2.0 unx      443 b- defN 16-Jan-01 00:00 pytokenizations-0.7.2.dist-info/METADATA
?rw-r--r--  2.0 unx      468 b- defN 16-Jan-01 00:00 pytokenizations-0.7.2.dist-info/RECORD
6 files, 1823 bytes uncompressed, 937 bytes compressed:  48.6%

[caltaojihun]

I think it's this line in poetry-core:

    def _add_file(self, wheel, full_path, rel_path):
        full_path, rel_path = str(full_path), str(rel_path)
        if os.sep != "/":
            # We always want to have /-separated paths in the zip file and in
            # RECORD
            rel_path = rel_path.replace(os.sep, "/")

        zinfo = zipfile.ZipInfo(rel_path)

When unspecified, zipfile.ZipInfo date_time defaults to 1980-01-01.

As for the 2016 dates, in the same file I found this:

        # The default is a fixed timestamp rather than the current time, so
        # that building a wheel twice on the same computer can automatically
        # give you the exact same result.
        date_time = (2016, 1, 1, 0, 0, 0)
        zi = zipfile.ZipInfo(rel_path, date_time)

It seems like this is intentional to create reproducible builds?

[mattsta]

oh, two places of weirdness. Thanks for finding the exact cause and effect!

If we want to use source file mtime dates in the zip file (for non-auto-generated files at least), looks like the fix would be:

file_date = datetime.datetime.fromtimestamp(os.stat(full_path).st_mtime).timetuple()[:-3]
zinfo = zipfile.ZipInfo(rel_path, file_date)

[mattsta]

and on second thought, instead of using fixed 1980 date for the _write_to_zip() calls, we could save the date of the most recently modified package source file then use it for metadata files too. Would maintain the same "double build guarantees same wheel output" invariant and also reflect reality a bit better.

...also now I see the building files are over in poetry-core but I had been only grepping here in poetry which is why I didn't find the ZipFile/ZipInfo usage. Sorry for throwing the issue under the wrong repo.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.