Introduce the spread app layout. #1431

jsirois · 2021-08-31T01:49:50Z

Beyond the top level pex and __main__.py files, the layout is an
implementation detail, but the current structure sheds some light on the
cache-friendly characteristics:

Given loose sources:

$ cat src/main.py
from pex.version import __version__; print(__version__)

A spread layout PEX looks like:

$ python -mpex pex -Dsrc -emain -opex.spread.venv --spread \
  --seed verbose --venv prepend | jq .
{
  "pex_root": "/home/jsirois/.pex",
  "python": "/usr/bin/python3.9",
  "pex": "/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex"
}
$ tree -a pex.spread.venv
pex.spread.venv
├── .bootstrap
├── .deps
│   └── pex-2.1.46-py2.py3-none-any.whl
├── __main__.py -> src/__main__.py
├── pex -> __main__.py
├── PEX-SPREAD-INFO
└── src
    ├── __main__.py
    ├── main.py
    └── PEX-INFO

And the runtime spreading is suggested by the new PEX-SPREAD-INFO
manifest:

{
  "sources": [
    "PEX-INFO",
    "__main__.py",
    "main.py"
  ],
  "spreads": [
    {
      "strip_zip_relpath": false,
      "unpack_relpath": "bootstraps/a54b6ae5e64e5b229388fdffc8adac141f3c416b",
      "zip_relpath": ".bootstrap"
    },
    {
      "strip_zip_relpath": true,
      "unpack_relpath": "installed_wheels/f627f0368a0e29be24aa8cadba74044b9ad990d7/pex-2.1.46-py2.py3-none-any.whl",
      "zip_relpath": ".deps/pex-2.1.46-py2.py3-none-any.whl"
    }
  ]
}

The layout adds new PEX_ROOT caches for the .bootstrap zip and
installed wheel chroot zips such that neither bootstraps - which are
tied to a version of Pex, nor installed wheel chroot zips, which are
constant for a given distribution version, are created more than once.

Closes #1424

Beyond the top level `pex` and `__main__.py` files, the layout is an implementation detail, but the current structure sheds some light on the cache-friendly characteristics: Given loose sources: ``` $ cat src/main.py from pex.version import __version__; print(__version__) ``` A spread layout PEX looks like: ``` $ python -mpex pex -Dsrc -emain -opex.spread.venv --spread \ --seed verbose --venv prepend | jq . { "pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex" } $ tree -a pex.spread.venv/ pex.spread.venv/ ├── .bootstrap ├── .deps │ └── pex-2.1.46-py2.py3-none-any.whl ├── __main__.py ├── pex -> __main__.py ├── PEX-SPREAD-INFO └── src ├── __main__.py ├── main.py └── PEX-INFO ``` And the runtime spreading is suggested by the new PEX-SPREAD-INFO manifest: ```json { "sources": [ "PEX-INFO", "__main__.py", "main.py" ], "spreads": [ { "strip_zip_relpath": false, "unpack_relpath": "bootstraps/a54b6ae5e64e5b229388fdffc8adac141f3c416b", "zip_relpath": ".bootstrap" }, { "strip_zip_relpath": true, "unpack_relpath": "installed_wheels/f627f0368a0e29be24aa8cadba74044b9ad990d7/pex-2.1.46-py2.py3-none-any.whl", "zip_relpath": ".deps/pex-2.1.46-py2.py3-none-any.whl" } ] } ``` The layout adds new PEX_ROOT caches for the `.bootstrap` zip and installed wheel chroot zips such that neither bootstraps - which are tied to a version of Pex, nor installed wheel chroot zips, which are constant for a given distribution version, are created more than once. Closes pex-tool#1424

jsirois · 2021-08-31T01:53:22Z

Space perf is ~perfect since there is 0 extra metadata or other files in the .bootstrap or wheel zips, but time perf is good too.

For this app:

$ cat src/main.py 
from pex.version import __version__; print(__version__)

$ python -mpex pex -Dsrc -emain -opex.pex --seed verbose; \
python -mpex pex -Dsrc -emain -opex.unzip --seed verbose --unzip; \
python -mpex pex -Dsrc -emain -opex.spread.unzip --spread --seed verbose; \
python -mpex pex -Dsrc -emain -opex.venv --seed verbose --venv prepend; \
python -mpex pex -Dsrc -emain -opex.spread.venv --spread --seed verbose --venv prepend
{"pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/dev/pantsbuild/jsirois-pex/pex.pex"}
{"pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/.pex/unzipped_pexes/55e9cd425cb1d261ce3782fec42420fd5edbbb91"}
{"pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/.pex/unzipped_pexes/55e9cd425cb1d261ce3782fec42420fd5edbbb91"}
{"pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex"}
{"pex_root": "/home/jsirois/.pex", "python": "/usr/bin/python3.9", "pex": "/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex"}

You find:

$ hyperfine -w2 ./pex.pex ./pex.unzip pex.spread.unzip/pex ./pex.venv pex.spread.venv/pex /home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex
Benchmark #1: ./pex.pex
  Time (mean ± σ):     596.2 ms ±   7.1 ms    [User: 548.0 ms, System: 45.7 ms]
  Range (min … max):   589.0 ms … 609.7 ms    10 runs
 
Benchmark #2: ./pex.unzip
  Time (mean ± σ):     344.2 ms ±   3.1 ms    [User: 299.2 ms, System: 44.1 ms]
  Range (min … max):   339.6 ms … 350.4 ms    10 runs
 
Benchmark #3: pex.spread.unzip/pex
  Time (mean ± σ):     401.5 ms ±   2.7 ms    [User: 349.8 ms, System: 50.3 ms]
  Range (min … max):   398.0 ms … 407.8 ms    10 runs
 
Benchmark #4: ./pex.venv
  Time (mean ± σ):      84.5 ms ±   1.7 ms    [User: 73.4 ms, System: 10.5 ms]
  Range (min … max):    82.0 ms …  90.5 ms    34 runs
 
Benchmark #5: pex.spread.venv/pex
  Time (mean ± σ):      75.1 ms ±   1.5 ms    [User: 64.2 ms, System: 10.5 ms]
  Range (min … max):    73.1 ms …  79.0 ms    36 runs
 
Benchmark #6: /home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex
  Time (mean ± σ):      23.7 ms ±   0.8 ms    [User: 20.3 ms, System: 3.4 ms]
  Range (min … max):    22.3 ms …  26.5 ms    109 runs
 
Summary
  '/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex' ran
    3.17 ± 0.13 times faster than 'pex.spread.venv/pex'
    3.57 ± 0.14 times faster than './pex.venv'
   14.53 ± 0.52 times faster than './pex.unzip'
   16.95 ± 0.60 times faster than 'pex.spread.unzip/pex'
   25.17 ± 0.93 times faster than './pex.pex'

This had no noticeable impact on performance.

This added ~60 ms to bootstrap with ~30ms going to import time and ~30ms going to attr dynamics.

jsirois · 2021-08-31T16:41:01Z

The niceties can be nasty. The one perf anomaly above was plain (unzip) --spread mode, which is slower than straight --unzip mode despite sharing a cache in ~/.pex/unzipped_pexes. Well, killing use of @attr.s in the spread.py file saved ~60ms - ~30ms for importing the thing and ~30ms for it to apply its runtime magic to two classes. Fix coming with updated perfs.

…/1424

jsirois · 2021-08-31T16:45:50Z

And now spread is faster for both venv and unzip cases which is to be expected, since there is no unzipping to do to read __main__.py when spread:

$ hyperfine -w2 ./pex.pex ./pex.unzip pex.spread.unzip/pex ./pex.venv pex.spread.venv/pex /home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex
Benchmark #1: ./pex.pex
  Time (mean ± σ):     592.3 ms ±   4.9 ms    [User: 548.1 ms, System: 42.5 ms]
  Range (min … max):   586.3 ms … 601.0 ms    10 runs
 
Benchmark #2: ./pex.unzip
  Time (mean ± σ):     344.8 ms ±   3.4 ms    [User: 303.8 ms, System: 39.9 ms]
  Range (min … max):   339.4 ms … 349.2 ms    10 runs
 
Benchmark #3: pex.spread.unzip/pex
  Time (mean ± σ):     337.4 ms ±   2.9 ms    [User: 298.8 ms, System: 37.4 ms]
  Range (min … max):   333.1 ms … 342.6 ms    10 runs
 
Benchmark #4: ./pex.venv
  Time (mean ± σ):      83.7 ms ±   1.2 ms    [User: 71.9 ms, System: 11.3 ms]
  Range (min … max):    82.0 ms …  88.5 ms    35 runs
 
Benchmark #5: pex.spread.venv/pex
  Time (mean ± σ):      75.1 ms ±   1.1 ms    [User: 65.0 ms, System: 9.5 ms]
  Range (min … max):    73.6 ms …  78.8 ms    38 runs
 
Benchmark #6: /home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex
  Time (mean ± σ):      23.9 ms ±   1.2 ms    [User: 20.3 ms, System: 3.5 ms]
  Range (min … max):    22.4 ms …  28.7 ms    108 runs
 
Summary
  '/home/jsirois/.pex/venvs/d2f743b9c1ebb156f794419c01b3422653cbdb61/2d1d404c3de23b1810386195be7410700b1feb14/pex' ran
    3.15 ± 0.16 times faster than 'pex.spread.venv/pex'
    3.50 ± 0.18 times faster than './pex.venv'
   14.14 ± 0.69 times faster than 'pex.spread.unzip/pex'
   14.44 ± 0.71 times faster than './pex.unzip'
   24.81 ± 1.22 times faster than './pex.pex'

…/1424

jsirois · 2021-09-01T20:05:22Z

@stuhood and @Eric-Arellano - friendly ping. As things stand, the Pants workarounds are still easy to cleanly revert and replace with just a --spread arg added for internal_only Pexes; so unless there are objections to this approach - it would be good to get this acceptable for landing.

stuhood · 2021-09-01T22:03:10Z

@stuhood and @Eric-Arellano - friendly ping. As things stand, the Pants workarounds are still easy to cleanly revert and replace with just a --spread arg added for internal_only Pexes; so unless there are objections to this approach - it would be good to get this acceptable for landing.

Will take a look today, sorry.

This seems like it will be useful for the repository.pex, but I didn't think that this was likely to replace using the PEX path, since using the PEX path makes the per-subset builds nearly-noops. Will need to run the numbers on that.

jsirois · 2021-09-01T22:54:55Z

This seems like it will be useful for the repository.pex, but I didn't think that this was likely to replace using the PEX path, since using the PEX path makes the per-subset builds nearly-noops. Will need to run the numbers on that.

When you review you should come to the understanding this makes the per-subset builds nearly-noops too. You get it all in one.
I'll add a subset CLI perf test here as a comment.

stuhood · 2021-09-01T23:00:48Z

The relevant set of steps that we're looking at is:

build repository.pex from R (for example: 130) requirements
build N (for example: 90) subsets of the repository.pex

So the relevant performance comparison is for all of the invokes in there. When composing via the PEX_PATH, step 2 has two halves:

2.a. build a single-entry PEX for each of R root requirements
2.b. compose the single-entry PEXes for the requirements that are relevant to each subset

2.a. takes time proportional to R, but 2.b. creates and captures empty 400kb PEXes (rather than copying and re-capturing all of the inputs).

I expect that which of --spread or using the PEX_PATH will be faster will be dependent on N and R, but as I said: haven't run the numbers.

jsirois · 2021-09-01T23:03:51Z

Stu, step 1 used to zip up the whole repository pex. but in --spead mode repository pex creation creates N +1 zips (+1 is for .bootstrap). The wall time of creating the 1 zip vs the N is ~ the same. So when you get to step 2a, that work is already done and sitting around in named_caches.

stuhood · 2021-09-01T23:12:10Z

Stu, step 1 used to zip up the whole repository pex. but in --spead mode repository pex creation creates N +1 zips (+1 is for .bootstrap). The wall time of creating the 1 zip vs the N is ~ the same. So when you get to step 2a, that work is already done and sitting around in named_caches.

You're using a different meaning of N than I am, which might confuse things. If we could continue to use N for "number of distinct subsets of the repository.pex", that would help.

You're using N to mean "number of requirements": let's use R for that instead? I'll update my comment.

jsirois · 2021-09-01T23:44:57Z

OK.

For the creation part, this current impl, which doesn't attempt to parallelize zip creation, repository.pex creation time using a resolve from toolchain that is R=125 at 44MB:

$ hyperfine -r5 -w1 -p "rm -rf ~/.pex repository.pex repository.pex.spread" "pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex" "pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex.spread --spread"
Benchmark #1: pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex
  Time (mean ± σ):     31.999 s ±  0.761 s    [User: 121.566 s, System: 17.429 s]
  Range (min … max):   30.785 s … 32.737 s    5 runs
 
Benchmark #2: pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex.spread --spread
  Time (mean ± σ):     32.695 s ±  0.529 s    [User: 126.904 s, System: 17.925 s]
  Range (min … max):   32.320 s … 33.627 s    5 runs
 
Summary
  'pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex' ran
    1.02 ± 0.03 times faster than 'pex.with-spread.pex -r 3rdparty/python/requirements.txt -f find-links --no-pypi -orepository.pex.spread --spread'

So slightly slower to create the spread repository pex with all the pre-seeded installed wheel chroots + paired cached zips.

I need to do a bit more work to get the N measurement, I'll be back with that.

stuhood · 2021-09-01T23:59:26Z

So slightly slower to create the spread repository pex with all the pre-seeded installed wheel chroots + paired cached zips.

I'll continue evaluating this after dinner, but for 2.b. (note: not 2.a., will follow up on that) the worst case when composing R==133 requirement PEXes into one subset using the PEX_PATH looks like:

$ wc -l transitive-pexes.txt
     133 transitive-pexes.txt
$ time ~/src/venvs/pex-2.1.45/bin/pex --pex-path=`cat transitive-pexes.txt | tr '\n' ':'` -o transitive-pex-path.pex

real	0m0.376s
user	0m0.282s
sys	0m0.091s

You then also don't need to compare/capture any of the transitive PEXes from the sandbox, because they were only inputs: not outputs.

2.a. definitely takes a non-trivial amount of time though: will follow up tonight.

jsirois · 2021-09-02T00:25:22Z

Here's the worst case using my sample which is R=125 again:

$ time pex.with-spread.pex -v -r 3rdparty/python/requirements.txt --pex-repository repository.pex.spread -obig.spread.pex --spread
pex: Building pex: 1111.6ms                                                                                                                  
pex:   Resolving distributions (['3rdparty/python/requirements.txt']): 1110.8ms
pex:     Resolving requirements from PEX repository.pex.spread.: 474.9ms
Saving PEX file to big.spread.pex
pex: Creating Spread PEX.: 10.7ms

real	0m1.383s
user	0m1.194s
sys	0m0.184s

Vs a small one:

$ time pex.with-spread.pex -v pytest --pex-repository repository.pex.spread -opytest.spread.pex --spread
pex: Building pex: 121.4ms                                                                                         
pex:   Resolving distributions (['pytest']): 120.8ms
pex:     Resolving requirements from PEX repository.pex.spread.: 111.1ms
Saving PEX file to pytest.spread.pex
pex: Creating Spread PEX.: 5.3ms

real	0m0.383s
user	0m0.352s
sys	0m0.031s

So that's 2a + 2b for --spread since --spread does 2a ahead of time when it builds the repository PEX. Note also that this includes a real resolve and that takes most of the time. Creating the spread is roughly constant time (10ms vs 5ms) given the big spread is 1600X bigger in this example:

$ du -sh big.spread.pex pytest.spread.pex
47M	big.spread.pex
28K	pytest.spread.pex

jsirois · 2021-09-02T00:27:18Z

All this is not even getting into the fact that the PEX_PATH trick blows the cache on every Pex upgrade. Here the installed wheel chroots are cached directy and so live forever unperturbed by Pex upgrades or any other muddying of the hash.

jsirois · 2021-09-02T00:32:19Z

IIUC you also don't need to capture any outputs from --spread step 2 either. Or you do, but they should be noops for the .bootstrap zip and all the installed wheel chroot zips since they are identical to the same zips captured when the repository spread pex was built. So ... unless capture of a file that has the same digest as one already captured is expensive, the local CAS should get a hit on the hash and noop. The cost is just hashing the file to determine "already captured / already in CAS".

jsirois · 2021-09-02T00:41:06Z

Also, IIUC the current PEX_PATH stuff isn't accounting for this time, which will be time it needs to spend to do 2b right. All that time is reflected in my --spread measurements above, but a bit faster because its in-process:

$ time pex-tools repository.pex repository info -v
{"project_name": "Django", "version": "3.2.7", "requires_python": ">=3.6", "requires_dists": ["asgiref<4,>=3.3.2", "pytz", "sqlparse>=0.2.2", "argon2-cffi>=19.1.0; extra == \"argon2\"", "bcrypt; extra == \"bcrypt\""], "location": "/home/jsirois/.pex/installed_wheels/bd2869794f67e9fa3262ec0ea6123f8d412e1280/Django-3.2.7-py3-none-any.whl"}
{"project_name": "asgiref", "version": "3.4.1", "requires_python": ">=3.6", "requires_dists": ["typing-extensions; python_version < \"3.8\"", "pytest; extra == \"tests\"", "pytest-asyncio; extra == \"tests\"", "mypy>=0.800; extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/eb64b48e7eef2d36868783a5921bbf37f9e9877d/asgiref-3.4.1-py3-none-any.whl"}
{"project_name": "pytz", "version": "2021.1", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/7863c68df502e8f7ff22d3bdf81aae8d0e558c15/pytz-2021.1-py2.py3-none-any.whl"}
{"project_name": "sqlparse", "version": "0.4.1", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/31a824f03e6361b07f3361d73d252f873809a1d9/sqlparse-0.4.1-py3-none-any.whl"}
{"project_name": "Faker", "version": "8.12.1", "requires_python": ">=3.6", "requires_dists": ["python-dateutil>=2.4", "text-unidecode==1.3"], "location": "/home/jsirois/.pex/installed_wheels/78fe37cb312e4852324210f5f40560adccbf3a73/Faker-8.12.1-py3-none-any.whl"}
{"project_name": "python-dateutil", "version": "2.8.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7", "requires_dists": ["six>=1.5"], "location": "/home/jsirois/.pex/installed_wheels/2a47a350d5496e313e81c8faedfff0f4bfb08497/python_dateutil-2.8.2-py2.py3-none-any.whl"}
{"project_name": "six", "version": "1.16.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/547dcc25df24fe530a87948c2cc1e2bd1fdba3d8/six-1.16.0-py2.py3-none-any.whl"}
{"project_name": "text-unidecode", "version": "1.3", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/1d5b45d76086e2bcc4d937ee3ea4223c7819ceb9/text_unidecode-1.3-py2.py3-none-any.whl"}
{"project_name": "PyYAML", "version": "5.4.1", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/038492df277775a5cc7270a0ff2876b812bc4dc2/PyYAML-5.4.1-cp39-cp39-manylinux1_x86_64.whl"}
{"project_name": "ansicolors", "version": "1.1.8", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/89bce2ecc91fc06f230d4812835389751204e123/ansicolors-1.1.8-py2.py3-none-any.whl"}
{"project_name": "beautifulsoup4", "version": "4.9.3", "requires_python": null, "requires_dists": ["soupsieve<2.0,>1.2; python_version < \"3.0\"", "soupsieve>1.2; python_version >= \"3.0\"", "html5lib; extra == \"html5lib\"", "lxml; extra == \"lxml\""], "location": "/home/jsirois/.pex/installed_wheels/9766fba9d62a4310b39c70fd6625bc496c39bbf5/beautifulsoup4-4.9.3-py3-none-any.whl"}
{"project_name": "soupsieve", "version": "2.2.1", "requires_python": ">=3.6", "requires_dists": ["backports.functools-lru-cache; python_version < \"3\""], "location": "/home/jsirois/.pex/installed_wheels/1d753c7f18a80cb17988817c1b96f3f64f93aaf5/soupsieve-2.2.1-py3-none-any.whl"}
{"project_name": "boto3", "version": "1.18.33", "requires_python": ">=3.6", "requires_dists": ["botocore<1.22.0,>=1.21.33", "jmespath<1.0.0,>=0.7.1", "s3transfer<0.6.0,>=0.5.0", "botocore[crt]<2.0a0,>=1.21.0; extra == \"crt\""], "location": "/home/jsirois/.pex/installed_wheels/30f659dcb9f87519a0e78bafd7c092681fb32a69/boto3-1.18.33-py3-none-any.whl"}
{"project_name": "botocore", "version": "1.21.33", "requires_python": ">=3.6", "requires_dists": ["jmespath<1.0.0,>=0.7.1", "python-dateutil<3.0.0,>=2.1", "urllib3<1.27,>=1.25.4", "awscrt==0.11.24; extra == \"crt\""], "location": "/home/jsirois/.pex/installed_wheels/29d212d79142ad8f7392bd8a827d0195ec085d38/botocore-1.21.33-py3-none-any.whl"}
{"project_name": "jmespath", "version": "0.10.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/0437ad2a566465b24e6c14f6d7805409ef58078c/jmespath-0.10.0-py2.py3-none-any.whl"}
{"project_name": "urllib3", "version": "1.26.6", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<4,>=2.7", "requires_dists": ["brotlipy>=0.6.0; extra == \"brotli\"", "pyOpenSSL>=0.14; extra == \"secure\"", "cryptography>=1.3.4; extra == \"secure\"", "idna>=2.0.0; extra == \"secure\"", "certifi; extra == \"secure\"", "ipaddress; python_version == \"2.7\" and extra == \"secure\"", "PySocks!=1.5.7,<2.0,>=1.5.6; extra == \"socks\""], "location": "/home/jsirois/.pex/installed_wheels/a9e23fddd8511cebabf7b3cadb0c59f915091497/urllib3-1.26.6-py2.py3-none-any.whl"}
{"project_name": "s3transfer", "version": "0.5.0", "requires_python": ">=3.6", "requires_dists": ["botocore<2.0a.0,>=1.12.36", "botocore[crt]<2.0a.0,>=1.20.29; extra == \"crt\""], "location": "/home/jsirois/.pex/installed_wheels/17438ec180e9f9dc464cd63b3e0abbfc2efc96c1/s3transfer-0.5.0-py3-none-any.whl"}
{"project_name": "chardet", "version": "4.0.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/dc2d9bec078c7e16c3c986b832154371d1635f4c/chardet-4.0.0-py2.py3-none-any.whl"}
{"project_name": "cryptography", "version": "3.4.8", "requires_python": ">=3.6", "requires_dists": ["cffi>=1.12", "sphinx!=1.8.0,!=3.1.0,!=3.1.1,>=1.6.5; extra == \"docs\"", "sphinx-rtd-theme; extra == \"docs\"", "doc8; extra == \"docstest\"", "pyenchant>=1.6.11; extra == \"docstest\"", "twine>=1.12.0; extra == \"docstest\"", "sphinxcontrib-spelling>=4.0.1; extra == \"docstest\"", "black; extra == \"pep8test\"", "flake8; extra == \"pep8test\"", "flake8-import-order; extra == \"pep8test\"", "pep8-naming; extra == \"pep8test\"", "setuptools-rust>=0.11.4; extra == \"sdist\"", "bcrypt>=3.1.5; extra == \"ssh\"", "pytest>=6.0; extra == \"test\"", "pytest-cov; extra == \"test\"", "pytest-subtests; extra == \"test\"", "pytest-xdist; extra == \"test\"", "pretend; extra == \"test\"", "iso8601; extra == \"test\"", "pytz; extra == \"test\"", "hypothesis!=3.79.2,>=1.11.4; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/3eae1d79d219a6722620797dc8bddaf1a55ead58/cryptography-3.4.8-cp36-abi3-manylinux_2_24_x86_64.whl"}
{"project_name": "cffi", "version": "1.14.6", "requires_python": null, "requires_dists": ["pycparser"], "location": "/home/jsirois/.pex/installed_wheels/55bc1fe4f1616d7c945873a622b8f805bd93da86/cffi-1.14.6-cp39-cp39-manylinux1_x86_64.whl"}
{"project_name": "pycparser", "version": "2.20", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/c30c9428329c45c02ca784b60dd675a6370fceac/pycparser-2.20-py2.py3-none-any.whl"}
{"project_name": "defusedxml", "version": "0.7.1", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/6e4fa7973bad171d72a8c72c5ff66deb08501075/defusedxml-0.7.1-py2.py3-none-any.whl"}
{"project_name": "django-extensions", "version": "3.1.3", "requires_python": ">=3.6", "requires_dists": ["Django>=2.2"], "location": "/home/jsirois/.pex/installed_wheels/6d9d0f4181a5ff0d0ced0e6acbe8ecfef41dc852/django_extensions-3.1.3-py3-none-any.whl"}
{"project_name": "django-prometheus", "version": "2.1.0", "requires_python": null, "requires_dists": ["prometheus-client>=0.7"], "location": "/home/jsirois/.pex/installed_wheels/f2633b9f73c97f7bdb15b6c595750c25fa75656a/django_prometheus-2.1.0-py2.py3-none-any.whl"}
{"project_name": "prometheus-client", "version": "0.11.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": ["twisted; extra == \"twisted\""], "location": "/home/jsirois/.pex/installed_wheels/b6618e1b58606cc64cc23a007376b70fe97cf1d7/prometheus_client-0.11.0-py2.py3-none-any.whl"}
{"project_name": "django-csp", "version": "3.7", "requires_python": null, "requires_dists": ["Django>=1.8", "jinja2>=2.9.6; extra == \"jinja2\"", "pytest<4.0; extra == \"tests\"", "pytest-django; extra == \"tests\"", "pytest-flakes==1.0.1; extra == \"tests\"", "pytest-pep8==1.0.6; extra == \"tests\"", "pep8==1.4.6; extra == \"tests\"", "mock==1.0.1; extra == \"tests\"", "six==1.12.0; extra == \"tests\"", "jinja2>=2.9.6; extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/366d016d6f78c2234755846ea171ab3ac0523ae9/django_csp-3.7-py2.py3-none-any.whl"}
{"project_name": "djangorestframework", "version": "3.12.4", "requires_python": ">=3.5", "requires_dists": ["django>=2.2"], "location": "/home/jsirois/.pex/installed_wheels/9893b27a05368a6654dd9234babc2c168c976e94/djangorestframework-3.12.4-py3-none-any.whl"}
{"project_name": "djproxy", "version": "2.3.6", "requires_python": null, "requires_dists": ["requests>=1.0.0", "django>=1.11", "six>=1.9.0"], "location": "/home/jsirois/.pex/installed_wheels/4d48e2869d996ea17ead08b658fd0bd6165a082e/djproxy-2.3.6-py2.py3-none-any.whl"}
{"project_name": "requests", "version": "2.26.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7", "requires_dists": ["urllib3<1.27,>=1.21.1", "certifi>=2017.4.17", "chardet<5,>=3.0.2; python_version < \"3\"", "idna<3,>=2.5; python_version < \"3\"", "charset-normalizer~=2.0.0; python_version >= \"3\"", "idna<4,>=2.5; python_version >= \"3\"", "PySocks!=1.5.7,>=1.5.6; extra == \"socks\"", "win-inet-pton; (sys_platform == \"win32\" and python_version == \"2.7\") and extra == \"socks\"", "chardet<5,>=3.0.2; extra == \"use_chardet_on_py3\""], "location": "/home/jsirois/.pex/installed_wheels/f5801320a7222b44c83d8bb823e927cde559dc80/requests-2.26.0-py2.py3-none-any.whl"}
{"project_name": "certifi", "version": "2021.5.30", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/fc68839fd0d50ba9bf79238d6d21f4d7ee840009/certifi-2021.5.30-py2.py3-none-any.whl"}
{"project_name": "charset-normalizer", "version": "2.0.4", "requires_python": ">=3.5.0", "requires_dists": ["unicodedata2; extra == \"unicode_backport\""], "location": "/home/jsirois/.pex/installed_wheels/d8fa24cd34e705bba93c78ae41aed16033f15a11/charset_normalizer-2.0.4-py3-none-any.whl"}
{"project_name": "idna", "version": "3.2", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/2654a7cea921439b3a5420339d0981c51c083781/idna-3.2-py3-none-any.whl"}
{"project_name": "docker", "version": "5.0.2", "requires_python": ">=3.6", "requires_dists": ["websocket-client>=0.32.0", "requests!=2.18.0,>=2.14.2", "pywin32==227; sys_platform == \"win32\"", "paramiko>=2.4.2; extra == \"ssh\"", "pyOpenSSL>=17.5.0; extra == \"tls\"", "cryptography>=3.4.7; extra == \"tls\"", "idna>=2.0.0; extra == \"tls\""], "location": "/home/jsirois/.pex/installed_wheels/8668061eb99fb5f3cba867e6db7aff7a9b2046ad/docker-5.0.2-py2.py3-none-any.whl"}
{"project_name": "websocket-client", "version": "1.2.1", "requires_python": ">=3.6", "requires_dists": ["python-socks; extra == \"optional\"", "wsaccel; extra == \"optional\"", "websockets; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/63619af5ed432a19975119f8fc84630a160d561e/websocket_client-1.2.1-py2.py3-none-any.whl"}
{"project_name": "drf-nested-routers", "version": "0.93.3", "requires_python": ">=3.5", "requires_dists": ["djangorestframework>=3.6.0", "Django>=1.11"], "location": "/home/jsirois/.pex/installed_wheels/8a5214b760fe4cef850b0862830cbd70026fae2c/drf_nested_routers-0.93.3-py2.py3-none-any.whl"}
{"project_name": "duo-universal", "version": "2.0.1", "requires_python": ">=3", "requires_dists": ["cryptography>=3.2", "PyJWT>=2.0", "pyOpenSSL>=19.0.0", "requests>=2.22.0", "wheel>=0.35.1"], "location": "/home/jsirois/.pex/installed_wheels/55180078a435b60061bbb3974da215800f51f877/duo_universal-2.0.1-py2.py3-none-any.whl"}
{"project_name": "PyJWT", "version": "2.1.0", "requires_python": ">=3.6", "requires_dists": ["cryptography<4.0.0,>=3.3.1; extra == \"crypto\"", "sphinx; extra == \"dev\"", "sphinx-rtd-theme; extra == \"dev\"", "zope.interface; extra == \"dev\"", "cryptography<4.0.0,>=3.3.1; extra == \"dev\"", "pytest<7.0.0,>=6.0.0; extra == \"dev\"", "coverage[toml]==5.0.4; extra == \"dev\"", "mypy; extra == \"dev\"", "pre-commit; extra == \"dev\"", "sphinx; extra == \"docs\"", "sphinx-rtd-theme; extra == \"docs\"", "zope.interface; extra == \"docs\"", "pytest<7.0.0,>=6.0.0; extra == \"tests\"", "coverage[toml]==5.0.4; extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/6314f5db6c034de33ef65f10dc87f6b2f2c0da89/PyJWT-2.1.0-py3-none-any.whl"}
{"project_name": "pyOpenSSL", "version": "20.0.1", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["cryptography>=3.2", "six>=1.5.2", "sphinx; extra == \"docs\"", "sphinx-rtd-theme; extra == \"docs\"", "flaky; extra == \"test\"", "pretend; extra == \"test\"", "pytest>=3.0.1; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/78e8cbfc1cf34da780d85da6e95987a0ea4a8ae5/pyOpenSSL-20.0.1-py2.py3-none-any.whl"}
{"project_name": "wheel", "version": "0.36.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["pytest>=3.0.0; extra == \"test\"", "pytest-cov; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/289865d703a0de9530641cd8fe3f6df18e51dfea/wheel-0.36.2-py2.py3-none-any.whl"}
{"project_name": "duo-web", "version": "1.3.0", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/f65fd8a2d571342315e2f76e70b4e200010b767b/duo_web-1.3.0-py2.py3-none-any.whl"}
{"project_name": "elasticsearch-dsl", "version": "7.4.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": ["six", "python-dateutil", "elasticsearch<8.0.0,>=7.0.0", "ipaddress; python_version < \"3.3\"", "mock; extra == \"develop\"", "pytest>=3.0.0; extra == \"develop\"", "pytest-cov; extra == \"develop\"", "pytest-mock<3.0.0; extra == \"develop\"", "pytz; extra == \"develop\"", "coverage<5.0.0; extra == \"develop\"", "sphinx; extra == \"develop\"", "sphinx-rtd-theme; extra == \"develop\""], "location": "/home/jsirois/.pex/installed_wheels/73e3707e2bb37b5223180c0730b2675aad3312b1/elasticsearch_dsl-7.4.0-py2.py3-none-any.whl"}
{"project_name": "elasticsearch", "version": "7.5.1", "requires_python": null, "requires_dists": ["urllib3>=1.21.1", "requests<3.0.0,>=2.0.0; extra == \"develop\"", "nose; extra == \"develop\"", "coverage; extra == \"develop\"", "mock; extra == \"develop\"", "pyaml; extra == \"develop\"", "nosexcover; extra == \"develop\"", "sphinx<1.7; extra == \"develop\"", "sphinx-rtd-theme; extra == \"develop\"", "black; extra == \"develop\"", "jinja2; extra == \"develop\"", "sphinx<1.7; extra == \"docs\"", "sphinx-rtd-theme; extra == \"docs\"", "requests<3.0.0,>=2.4.0; extra == \"requests\""], "location": "/home/jsirois/.pex/installed_wheels/1815529bd3fa89a2c14edcbf5025edc2372a3aad/elasticsearch-7.5.1-py2.py3-none-any.whl"}
{"project_name": "fasteners", "version": "0.16", "requires_python": null, "requires_dists": ["six", "monotonic>=0.1; python_version < \"3.4\""], "location": "/home/jsirois/.pex/installed_wheels/b2603b37f25f70972f79a24f2ed92cb272a46259/fasteners-0.16-py2.py3-none-any.whl"}
{"project_name": "freezegun", "version": "1.1.0", "requires_python": ">=3.5", "requires_dists": ["python-dateutil>=2.7"], "location": "/home/jsirois/.pex/installed_wheels/a357b1a42ab67f8bbff2dd52d1d5e3cad31248b8/freezegun-1.1.0-py2.py3-none-any.whl"}
{"project_name": "GitPython", "version": "3.1.18", "requires_python": ">=3.6", "requires_dists": ["gitdb<5,>=4.0.1", "typing-extensions>=3.7.4.0; python_version < \"3.8\""], "location": "/home/jsirois/.pex/installed_wheels/d514586e0526201f22e63060ac38be0486a99483/GitPython-3.1.18-py3-none-any.whl"}
{"project_name": "gitdb", "version": "4.0.7", "requires_python": ">=3.4", "requires_dists": ["smmap<5,>=3.0.1"], "location": "/home/jsirois/.pex/installed_wheels/e565d26201f94c1562760e391272f5e1be86e29f/gitdb-4.0.7-py3-none-any.whl"}
{"project_name": "smmap", "version": "4.0.0", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/af7fee19d8d6e8e9ea6d7ca16a515bb4a989f25a/smmap-4.0.0-py2.py3-none-any.whl"}
{"project_name": "gunicorn", "version": "20.1.0", "requires_python": ">=3.5", "requires_dists": ["setuptools>=3.0", "eventlet>=0.24.1; extra == \"eventlet\"", "gevent>=1.4.0; extra == \"gevent\"", "setproctitle; extra == \"setproctitle\"", "tornado>=0.2; extra == \"tornado\""], "location": "/home/jsirois/.pex/installed_wheels/9dd453e646378477913323592c8ef842e13a010f/gunicorn-20.1.0-py3-none-any.whl"}
{"project_name": "setuptools", "version": "56.2.0", "requires_python": ">=3.6", "requires_dists": ["certifi==2016.9.26; extra == \"certs\"", "sphinx; extra == \"docs\"", "jaraco.packaging>=8.2; extra == \"docs\"", "rst.linker>=1.9; extra == \"docs\"", "pygments-github-lexers==0.0.5; extra == \"docs\"", "sphinx-inline-tabs; extra == \"docs\"", "wincertstore==0.2; sys_platform == \"win32\" and extra == \"ssl\"", "pytest>=4.6; extra == \"testing\"", "pytest-checkdocs>=2.4; extra == \"testing\"", "pytest-flake8; extra == \"testing\"", "pytest-cov; extra == \"testing\"", "pytest-enabler>=1.0.1; extra == \"testing\"", "mock; extra == \"testing\"", "flake8-2020; extra == \"testing\"", "virtualenv>=13.0.0; extra == \"testing\"", "pytest-virtualenv>=1.2.7; extra == \"testing\"", "wheel; extra == \"testing\"", "paver; extra == \"testing\"", "pip>=19.1; extra == \"testing\"", "jaraco.envs; extra == \"testing\"", "pytest-xdist; extra == \"testing\"", "sphinx; extra == \"testing\"", "jaraco.path>=3.2.0; extra == \"testing\"", "pytest-black>=0.3.7; (platform_python_implementation != \"PyPy\" and python_version < \"3.10\") and extra == \"testing\"", "pytest-mypy; (platform_python_implementation != \"PyPy\" and python_version < \"3.10\") and extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/3d40b32509183f1795401af8c8ebc4b1df558b60/setuptools-56.2.0-py3-none-any.whl"}
{"project_name": "hdrhistogram", "version": "0.9.0", "requires_python": null, "requires_dists": ["pbr>=1.4", "future>=0.15.2"], "location": "/home/jsirois/.pex/installed_wheels/0ad9f30378a451591fc2c1111641140b22ee311b/hdrhistogram-0.9.0-cp39-cp39-linux_x86_64.whl"}
{"project_name": "pbr", "version": "5.6.0", "requires_python": ">=2.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/8a910dc5cd7c2af288b9429d14d08700167642bb/pbr-5.6.0-py2.py3-none-any.whl"}
{"project_name": "future", "version": "0.18.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/9cbf8eb69377dcc376ac4a7139e7b490ee0866f6/future-0.18.2-py3-none-any.whl"}
{"project_name": "httpx", "version": "0.19.0", "requires_python": ">=3.6", "requires_dists": ["certifi", "charset-normalizer", "sniffio", "rfc3986[idna2008]<2,>=1.3", "httpcore<0.14.0,>=0.13.3", "async-generator; python_version < \"3.7\"", "brotlicffi; platform_python_implementation != \"CPython\" and extra == \"brotli\"", "brotli; platform_python_implementation == \"CPython\" and extra == \"brotli\"", "h2<5,>=3; extra == \"http2\""], "location": "/home/jsirois/.pex/installed_wheels/3313a5b6860b5c4ce260d13fcd00c78bddf8005b/httpx-0.19.0-py3-none-any.whl"}
{"project_name": "sniffio", "version": "1.2.0", "requires_python": ">=3.5", "requires_dists": ["contextvars>=2.1; python_version < \"3.7\""], "location": "/home/jsirois/.pex/installed_wheels/aec1e7a08215d88264d95420f3141a6693e83a7d/sniffio-1.2.0-py3-none-any.whl"}
{"project_name": "rfc3986", "version": "1.5.0", "requires_python": null, "requires_dists": ["idna; extra == \"idna2008\""], "location": "/home/jsirois/.pex/installed_wheels/6e0fee64545345070abbab6df4b67ab94beb261b/rfc3986-1.5.0-py2.py3-none-any.whl"}
{"project_name": "httpcore", "version": "0.13.6", "requires_python": ">=3.6", "requires_dists": ["h11<0.13,>=0.11", "sniffio==1.*", "anyio==3.*", "h2<5,>=3; extra == \"http2\""], "location": "/home/jsirois/.pex/installed_wheels/20e14e95fdde5badbb5641b342e1f190e9a5850e/httpcore-0.13.6-py3-none-any.whl"}
{"project_name": "h11", "version": "0.12.0", "requires_python": ">=3.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/2a083951ef93374f7fa404ec6f5304b9fc55b677/h11-0.12.0-py3-none-any.whl"}
{"project_name": "anyio", "version": "3.3.0", "requires_python": ">=3.6.2", "requires_dists": ["idna>=2.8", "sniffio>=1.1", "dataclasses; python_version < \"3.7\"", "typing-extensions; python_version < \"3.8\"", "sphinx-rtd-theme; extra == \"doc\"", "sphinx-autodoc-typehints>=1.2.0; extra == \"doc\"", "coverage[toml]>=4.5; extra == \"test\"", "hypothesis>=4.0; extra == \"test\"", "pytest>=6.0; extra == \"test\"", "pytest-mock>=3.6.1; extra == \"test\"", "trustme; extra == \"test\"", "uvloop<0.15; (python_version < \"3.7\" and (platform_python_implementation == \"CPython\" and platform_system != \"Windows\")) and extra == \"test\"", "mock>=4; python_version < \"3.8\" and extra == \"test\"", "uvloop>=0.15; (python_version >= \"3.7\" and (platform_python_implementation == \"CPython\" and platform_system != \"Windows\")) and extra == \"test\"", "trio>=0.16; extra == \"trio\""], "location": "/home/jsirois/.pex/installed_wheels/aac82797d77cdecd3f55017fdcdf6f84796cd689/anyio-3.3.0-py3-none-any.whl"}
{"project_name": "humanize", "version": "3.11.0", "requires_python": ">=3.6", "requires_dists": ["setuptools", "freezegun; extra == \"tests\"", "pytest; extra == \"tests\"", "pytest-cov; extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/474f13510bf2626c6727c35b162befe29aaf0e15/humanize-3.11.0-py3-none-any.whl"}
{"project_name": "influxdb-client", "version": "1.20.0", "requires_python": ">=3.6", "requires_dists": ["rx>=3.0.1", "certifi>=14.05.14", "six>=1.10", "python-dateutil>=2.5.3", "setuptools>=21.0.0", "urllib3>=1.15.1", "pytz>=2019.1", "ciso8601>=2.1.1; extra == \"ciso\"", "pandas>=0.25.3; extra == \"extra\"", "numpy; extra == \"extra\"", "coverage>=4.0.3; extra == \"test\"", "nose>=1.3.7; extra == \"test\"", "pluggy>=0.3.1; extra == \"test\"", "py>=1.4.31; extra == \"test\"", "randomize>=0.13; extra == \"test\"", "pytest>=5.0.0; extra == \"test\"", "httpretty==1.0.5; extra == \"test\"", "psutil>=5.6.3; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/67a859d4bb25df34603d92673c9ea3e3479198ad/influxdb_client-1.20.0-py3-none-any.whl"}
{"project_name": "Rx", "version": "3.2.0", "requires_python": ">=3.6.0", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/960484ed1717311621523425a9cf4c23f5e681d5/Rx-3.2.0-py3-none-any.whl"}
{"project_name": "Jinja2", "version": "2.11.3", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["MarkupSafe>=0.23", "Babel>=0.8; extra == \"i18n\""], "location": "/home/jsirois/.pex/installed_wheels/f4b709b300b9f8294c68957ba0c8b8f0a6ec2fbb/Jinja2-2.11.3-py2.py3-none-any.whl"}
{"project_name": "MarkupSafe", "version": "2.0.1", "requires_python": ">=3.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/77ae16252f94b4a4dde1f89a98a16e84ca0b44c6/MarkupSafe-2.0.1-cp39-cp39-manylinux1_x86_64.manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_5_x86_64.whl"}
{"project_name": "jwcrypto", "version": "1.0", "requires_python": null, "requires_dists": ["cryptography>=2.3", "deprecated"], "location": "/home/jsirois/.pex/installed_wheels/a118baf8da19adfefe5de2be1d3331d163c80f7c/jwcrypto-1.0-py2.py3-none-any.whl"}
{"project_name": "Deprecated", "version": "1.2.12", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": ["wrapt<2,>=1.10", "tox; extra == \"dev\"", "bump2version<1; extra == \"dev\"", "sphinx<2; extra == \"dev\"", "importlib-metadata<3; python_version < \"3\" and extra == \"dev\"", "importlib-resources<4; python_version < \"3\" and extra == \"dev\"", "configparser<5; python_version < \"3\" and extra == \"dev\"", "sphinxcontrib-websupport<2; python_version < \"3\" and extra == \"dev\"", "zipp<2; python_version < \"3\" and extra == \"dev\"", "PyTest<5; python_version < \"3.6\" and extra == \"dev\"", "PyTest-Cov<2.6; python_version < \"3.6\" and extra == \"dev\"", "PyTest; python_version >= \"3.6\" and extra == \"dev\"", "PyTest-Cov; python_version >= \"3.6\" and extra == \"dev\""], "location": "/home/jsirois/.pex/installed_wheels/7d3bea62e0cf66be5856075938a02c65a7d122d5/Deprecated-1.2.12-py2.py3-none-any.whl"}
{"project_name": "wrapt", "version": "1.12.1", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/223080c0d99f96ddd2429d41cce7b641d7559725/wrapt-1.12.1-cp39-cp39-linux_x86_64.whl"}
{"project_name": "kubernetes", "version": "18.20.0", "requires_python": null, "requires_dists": ["certifi>=14.05.14", "six>=1.9.0", "python-dateutil>=2.5.3", "setuptools>=21.0.0", "pyyaml>=5.4.1", "google-auth>=1.0.1", "websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0", "requests", "requests-oauthlib", "urllib3>=1.24.2", "ipaddress>=1.0.17; python_version == \"2.7\"", "adal>=1.0.2; extra == \"adal\""], "location": "/home/jsirois/.pex/installed_wheels/47fc3f0dda364bb9c75088f3c1842c4f74c844b5/kubernetes-18.20.0-py2.py3-none-any.whl"}
{"project_name": "google-auth", "version": "2.0.2", "requires_python": ">=3.6", "requires_dists": ["cachetools<5.0,>=2.0.0", "pyasn1-modules>=0.2.1", "rsa<5,>=3.1.4", "setuptools>=40.3.0", "aiohttp<4.0.0dev,>=3.6.2; extra == \"aiohttp\"", "requests<3.0.0dev,>=2.20.0; extra == \"aiohttp\"", "pyopenssl>=20.0.0; extra == \"pyopenssl\"", "pyu2f>=0.1.5; extra == \"reauth\""], "location": "/home/jsirois/.pex/installed_wheels/fffcb184715691a3bae08855fb323907a79ab493/google_auth-2.0.2-py2.py3-none-any.whl"}
{"project_name": "cachetools", "version": "4.2.2", "requires_python": "~=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/01426b985ebaecf58eaf2b0b90795e91e66da49b/cachetools-4.2.2-py3-none-any.whl"}
{"project_name": "pyasn1-modules", "version": "0.2.8", "requires_python": null, "requires_dists": ["pyasn1<0.5.0,>=0.4.6"], "location": "/home/jsirois/.pex/installed_wheels/8f5ce8da1e595ee5a07bce5a3dd655df5f8a87dc/pyasn1_modules-0.2.8-py2.py3-none-any.whl"}
{"project_name": "pyasn1", "version": "0.4.8", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/0f8ec592c4bb4b222ae65b4f6dfd4d503fcbef4b/pyasn1-0.4.8-py2.py3-none-any.whl"}
{"project_name": "rsa", "version": "4.7.2", "requires_python": "<4,>=3.5", "requires_dists": ["pyasn1>=0.1.3"], "location": "/home/jsirois/.pex/installed_wheels/d92c754c195048c7b2f25cac364f02e3e6d2dc25/rsa-4.7.2-py3-none-any.whl"}
{"project_name": "requests-oauthlib", "version": "1.3.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": ["oauthlib>=3.0.0", "requests>=2.0.0", "oauthlib[signedtoken]>=3.0.0; extra == \"rsa\""], "location": "/home/jsirois/.pex/installed_wheels/993cb3bd21aa271611149d23be968b6d5ed94036/requests_oauthlib-1.3.0-py2.py3-none-any.whl"}
{"project_name": "oauthlib", "version": "3.1.1", "requires_python": ">=3.6", "requires_dists": ["cryptography<4,>=3.0.0; extra == \"rsa\"", "blinker>=1.4.0; extra == \"signals\"", "cryptography<4,>=3.0.0; extra == \"signedtoken\"", "pyjwt<3,>=2.0.0; extra == \"signedtoken\""], "location": "/home/jsirois/.pex/installed_wheels/79a969f17e8251d8503f1529408c1bb3458763aa/oauthlib-3.1.1-py2.py3-none-any.whl"}
{"project_name": "more-itertools", "version": "8.8.0", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/887f7ed215419136a84f15ceda7ee41b424c3900/more_itertools-8.8.0-py3-none-any.whl"}
{"project_name": "moto", "version": "2.2.6", "requires_python": null, "requires_dists": ["boto3>=1.9.201", "botocore>=1.12.201", "cryptography>=3.3.1", "requests>=2.5", "xmltodict", "werkzeug", "pytz", "python-dateutil<3.0.0,>=2.1", "responses>=0.9.0", "MarkupSafe!=2.0.0a1", "Jinja2>=2.10.1", "more-itertools", "importlib-metadata; python_version < \"3.8\"", "PyYAML>=5.1; extra == \"all\"", "python-jose[cryptography]<4.0.0,>=3.1.0; extra == \"all\"", "ecdsa<0.15; extra == \"all\"", "docker>=2.5.1; extra == \"all\"", "jsondiff>=1.1.2; extra == \"all\"", "aws-xray-sdk!=0.96,>=0.93; extra == \"all\"", "idna<3,>=2.5; extra == \"all\"", "cfn-lint>=0.4.0; extra == \"all\"", "sshpubkeys>=3.1.0; extra == \"all\"", "setuptools; extra == \"all\"", "python-jose[cryptography]<4.0.0,>=3.1.0; extra == \"apigateway\"", "ecdsa<0.15; extra == \"apigateway\"", "docker>=2.5.1; extra == \"awslambda\"", "docker>=2.5.1; extra == \"batch\"", "docker>=2.5.1; extra == \"cloudformation\"", "PyYAML>=5.1; extra == \"cloudformation\"", "cfn-lint>=0.4.0; extra == \"cloudformation\"", "python-jose[cryptography]<4.0.0,>=3.1.0; extra == \"cognitoidp\"", "ecdsa<0.15; extra == \"cognitoidp\"", "docker>=2.5.1; extra == \"dynamodb2\"", "docker>=2.5.1; extra == \"dynamodbstreams\"", "sshpubkeys>=3.1.0; extra == \"ec2\"", "sshpubkeys>=3.1.0; extra == \"efs\"", "jsondiff>=1.1.2; extra == \"iotdata\"", "PyYAML>=5.1; extra == \"s3\"", "PyYAML>=5.1; extra == \"server\"", "python-jose[cryptography]<4.0.0,>=3.1.0; extra == \"server\"", "ecdsa<0.15; extra == \"server\"", "docker>=2.5.1; extra == \"server\"", "jsondiff>=1.1.2; extra == \"server\"", "aws-xray-sdk!=0.96,>=0.93; extra == \"server\"", "idna<3,>=2.5; extra == \"server\"", "cfn-lint>=0.4.0; extra == \"server\"", "sshpubkeys>=3.1.0; extra == \"server\"", "setuptools; extra == \"server\"", "flask; extra == \"server\"", "flask-cors; extra == \"server\"", "PyYAML>=5.1; extra == \"ssm\"", "aws-xray-sdk!=0.96,>=0.93; extra == \"xray\"", "setuptools; extra == \"xray\""], "location": "/home/jsirois/.pex/installed_wheels/f0ab6a287a0eb1792f01c28d7adba7c142ee96ed/moto-2.2.6-py2.py3-none-any.whl"}
{"project_name": "xmltodict", "version": "0.12.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/17a5d32bfc2ad938280b13f9ac09ea2f02428acb/xmltodict-0.12.0-py2.py3-none-any.whl"}
{"project_name": "Werkzeug", "version": "2.0.1", "requires_python": ">=3.6", "requires_dists": ["dataclasses; python_version < \"3.7\"", "watchdog; extra == \"watchdog\""], "location": "/home/jsirois/.pex/installed_wheels/9926e5c7ca521ccca58e19f07b711eaa9d97311d/Werkzeug-2.0.1-py3-none-any.whl"}
{"project_name": "responses", "version": "0.13.4", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["requests>=2.0", "urllib3>=1.25.10", "six", "mock; python_version < \"3.3\"", "cookies; python_version < \"3.4\"", "coverage<6.0.0,>=3.7.1; extra == \"tests\"", "pytest-cov; extra == \"tests\"", "pytest-localserver; extra == \"tests\"", "flake8; extra == \"tests\"", "types-mock; extra == \"tests\"", "types-requests; extra == \"tests\"", "types-six; extra == \"tests\"", "pytest<5.0,>=4.6; python_version < \"3.5\" and extra == \"tests\"", "pytest>=4.6; python_version >= \"3.5\" and extra == \"tests\"", "mypy; python_version >= \"3.5\" and extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/65dab6080910c54d8aedbb355079f8175ff47e24/responses-0.13.4-py2.py3-none-any.whl"}
{"project_name": "sshpubkeys", "version": "3.3.1", "requires_python": ">=3", "requires_dists": ["cryptography>=2.1.4", "ecdsa>=0.13", "twine; extra == \"dev\"", "wheel; extra == \"dev\"", "yapf; extra == \"dev\""], "location": "/home/jsirois/.pex/installed_wheels/2ea2479eea8edb5d437f106ae1ad59a1ab7447b0/sshpubkeys-3.3.1-py2.py3-none-any.whl"}
{"project_name": "ecdsa", "version": "0.17.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.6", "requires_dists": ["six>=1.9.0", "gmpy; extra == \"gmpy\"", "gmpy2; extra == \"gmpy2\""], "location": "/home/jsirois/.pex/installed_wheels/f879fabe2e0b707ca9fa498bcfeac370306f737b/ecdsa-0.17.0-py2.py3-none-any.whl"}
{"project_name": "ndjson", "version": "0.3.1", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/01fd52e9c718232b0f36ea56bf2e3831b80fda7f/ndjson-0.3.1-py2.py3-none-any.whl"}
{"project_name": "networkx", "version": "2.6.2", "requires_python": ">=3.7", "requires_dists": ["numpy>=1.19; extra == \"default\"", "scipy!=1.6.1,>=1.5; extra == \"default\"", "matplotlib>=3.3; extra == \"default\"", "pandas>=1.1; extra == \"default\"", "black==21.5b1; extra == \"developer\"", "pre-commit>=2.12; extra == \"developer\"", "sphinx~=4.0; extra == \"doc\"", "pydata-sphinx-theme~=0.6; extra == \"doc\"", "sphinx-gallery~=0.9; extra == \"doc\"", "numpydoc>=1.1; extra == \"doc\"", "pillow>=8.2; extra == \"doc\"", "nb2plots>=0.6; extra == \"doc\"", "texext>=0.6.6; extra == \"doc\"", "lxml>=4.5; extra == \"extra\"", "pygraphviz>=1.7; extra == \"extra\"", "pydot>=1.4.1; extra == \"extra\"", "pytest>=6.2; extra == \"test\"", "pytest-cov>=2.12; extra == \"test\"", "codecov>=2.1; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/9474e36034cc0b99f1004b5a90b3e232244458f2/networkx-2.6.2-py3-none-any.whl"}
{"project_name": "packaging", "version": "21.0", "requires_python": ">=3.6", "requires_dists": ["pyparsing>=2.0.2"], "location": "/home/jsirois/.pex/installed_wheels/c0f8081a1319a172990fe3d320c6a1fa5a394ef4/packaging-21.0-py3-none-any.whl"}
{"project_name": "pyparsing", "version": "2.4.7", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/ecc567fe9970156990b834ca5a203422cb85ea2b/pyparsing-2.4.7-py2.py3-none-any.whl"}
{"project_name": "pdpyras", "version": "4.3.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=3.5", "requires_dists": ["requests", "urllib3"], "location": "/home/jsirois/.pex/installed_wheels/23621c0dffc26b8fde8686a2f8d2d696efa38548/pdpyras-4.3.0-py3-none-any.whl"}
{"project_name": "pex", "version": "2.1.47", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<3.10,>=2.7", "requires_dists": ["subprocess32>=3.2.7; extra == \"subprocess\" and python_version < \"3\""], "location": "/home/jsirois/.pex/installed_wheels/5b5c8238205ae22221e2dcda0b28c45b0a6fc4cd/pex-2.1.47-py2.py3-none-any.whl"}
{"project_name": "pkginfo", "version": "1.7.1", "requires_python": null, "requires_dists": ["nose; extra == \"testing\"", "coverage; extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/a92282dc8ff0186be3fc30bd782c593b0dc3b16b/pkginfo-1.7.1-py2.py3-none-any.whl"}
{"project_name": "plyvel", "version": "1.3.0", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/dcaaa9e4b1fcf467f8c0a9166247826d34c4d11d/plyvel-1.3.0-cp39-cp39-manylinux2010_x86_64.whl"}
{"project_name": "psycopg2-binary", "version": "2.9.1", "requires_python": ">=3.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/3b33c3a9d956e76cbe66b27dea7180193e308313/psycopg2_binary-2.9.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl"}
{"project_name": "Pygments", "version": "2.10.0", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/e03057794c2959bda5f4e47062bd4f1251d01d12/Pygments-2.10.0-py3-none-any.whl"}
{"project_name": "pytest-django", "version": "4.4.0", "requires_python": ">=3.5", "requires_dists": ["pytest>=5.4.0", "sphinx; extra == \"docs\"", "sphinx-rtd-theme; extra == \"docs\"", "Django; extra == \"testing\"", "django-configurations>=2.0; extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/28f17da13441f00176a3bd0bc321d51fa18760e7/pytest_django-4.4.0-py3-none-any.whl"}
{"project_name": "pytest", "version": "6.2.5", "requires_python": ">=3.6", "requires_dists": ["attrs>=19.2.0", "iniconfig", "packaging", "pluggy<2.0,>=0.12", "py>=1.8.2", "toml", "importlib-metadata>=0.12; python_version < \"3.8\"", "atomicwrites>=1.0; sys_platform == \"win32\"", "colorama; sys_platform == \"win32\"", "argcomplete; extra == \"testing\"", "hypothesis>=3.56; extra == \"testing\"", "mock; extra == \"testing\"", "nose; extra == \"testing\"", "requests; extra == \"testing\"", "xmlschema; extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/8757823695748bae1e0639d2acced4b53c33e5c4/pytest-6.2.5-py3-none-any.whl"}
{"project_name": "attrs", "version": "21.2.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["coverage[toml]>=5.0.2; extra == \"dev\"", "hypothesis; extra == \"dev\"", "pympler; extra == \"dev\"", "pytest>=4.3.0; extra == \"dev\"", "six; extra == \"dev\"", "mypy; extra == \"dev\"", "pytest-mypy-plugins; extra == \"dev\"", "zope.interface; extra == \"dev\"", "furo; extra == \"dev\"", "sphinx; extra == \"dev\"", "sphinx-notfound-page; extra == \"dev\"", "pre-commit; extra == \"dev\"", "furo; extra == \"docs\"", "sphinx; extra == \"docs\"", "zope.interface; extra == \"docs\"", "sphinx-notfound-page; extra == \"docs\"", "coverage[toml]>=5.0.2; extra == \"tests\"", "hypothesis; extra == \"tests\"", "pympler; extra == \"tests\"", "pytest>=4.3.0; extra == \"tests\"", "six; extra == \"tests\"", "mypy; extra == \"tests\"", "pytest-mypy-plugins; extra == \"tests\"", "zope.interface; extra == \"tests\"", "coverage[toml]>=5.0.2; extra == \"tests_no_zope\"", "hypothesis; extra == \"tests_no_zope\"", "pympler; extra == \"tests_no_zope\"", "pytest>=4.3.0; extra == \"tests_no_zope\"", "six; extra == \"tests_no_zope\"", "mypy; extra == \"tests_no_zope\"", "pytest-mypy-plugins; extra == \"tests_no_zope\""], "location": "/home/jsirois/.pex/installed_wheels/3abec4c7ac272adb8bb81d17f4f99978842ce8d8/attrs-21.2.0-py2.py3-none-any.whl"}
{"project_name": "iniconfig", "version": "1.1.1", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/a4e09ca437e4f19dfeea390e1b59f35e06eea86c/iniconfig-1.1.1-py2.py3-none-any.whl"}
{"project_name": "pluggy", "version": "1.0.0", "requires_python": ">=3.6", "requires_dists": ["importlib-metadata>=0.12; python_version < \"3.8\"", "pre-commit; extra == \"dev\"", "tox; extra == \"dev\"", "pytest; extra == \"testing\"", "pytest-benchmark; extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/465a9bc3d68a39da28eca70b6c9ec344bfb3a850/pluggy-1.0.0-py2.py3-none-any.whl"}
{"project_name": "py", "version": "1.10.0", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/ff9b628c6ad0d3f9d2c606e28ddf25723ee52d7c/py-1.10.0-py2.py3-none-any.whl"}
{"project_name": "toml", "version": "0.10.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,>=2.6", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/090d1b9a9c986d33e2c5abd4a878d7c325d28f27/toml-0.10.2-py2.py3-none-any.whl"}
{"project_name": "pytest-httpx", "version": "0.13.0", "requires_python": ">=3.6", "requires_dists": ["httpx==0.19.*", "pytest==6.*", "pytest-asyncio==0.15.*; extra == \"testing\"", "pytest-cov==2.*; extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/ca1f5c758d74d8281fca396e7737f9e9fe4022af/pytest_httpx-0.13.0-py3-none-any.whl"}
{"project_name": "pytest-responses", "version": "0.5.0", "requires_python": null, "requires_dists": ["responses", "pytest>=2.5", "flake8; extra == \"tests\""], "location": "/home/jsirois/.pex/installed_wheels/8bc05290edcd86a3dfc26858a8d9ee6981c9c4f1/pytest_responses-0.5.0-py2.py3-none-any.whl"}
{"project_name": "python-jose", "version": "3.3.0", "requires_python": null, "requires_dists": ["ecdsa!=0.15", "rsa", "pyasn1", "cryptography>=3.4.0; extra == \"cryptography\"", "pycrypto<2.7.0,>=2.6.0; extra == \"pycrypto\"", "pyasn1; extra == \"pycrypto\"", "pycryptodome<4.0.0,>=3.3.1; extra == \"pycryptodome\"", "pyasn1; extra == \"pycryptodome\""], "location": "/home/jsirois/.pex/installed_wheels/73c6e6464598c8048575cb942cda199174071149/python_jose-3.3.0-py2.py3-none-any.whl"}
{"project_name": "redis", "version": "3.5.3", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["hiredis>=0.1.3; extra == \"hiredis\""], "location": "/home/jsirois/.pex/installed_wheels/f30ef1aa2eea6064ff4c221f4e4b9286e662e3da/redis-3.5.3-py2.py3-none-any.whl"}
{"project_name": "requests-aws4auth", "version": "1.1.1", "requires_python": null, "requires_dists": ["requests", "six"], "location": "/home/jsirois/.pex/installed_wheels/4f02f249b184ee6316a289ed6e71634cde1a0af1/requests_aws4auth-1.1.1-py2.py3-none-any.whl"}
{"project_name": "rich", "version": "10.9.0", "requires_python": "<4.0,>=3.6", "requires_dists": ["colorama<0.5.0,>=0.4.0", "commonmark<0.10.0,>=0.9.0", "dataclasses<0.9,>=0.7; python_version >= \"3.6\" and python_version < \"3.7\"", "ipywidgets<8.0.0,>=7.5.1; extra == \"jupyter\"", "pygments<3.0.0,>=2.6.0", "typing-extensions<4.0.0,>=3.7.4; python_version < \"3.8\""], "location": "/home/jsirois/.pex/installed_wheels/ecc3fc1a17bcf496bb013de87fb03dade245ff53/rich-10.9.0-py3-none-any.whl"}
{"project_name": "colorama", "version": "0.4.4", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/3c60182c7891c668eadf254032466a1bfa9b4d63/colorama-0.4.4-py2.py3-none-any.whl"}
{"project_name": "commonmark", "version": "0.9.1", "requires_python": null, "requires_dists": ["future>=0.14.0; python_version < \"3\"", "flake8==3.7.8; extra == \"test\"", "hypothesis==3.55.3; extra == \"test\""], "location": "/home/jsirois/.pex/installed_wheels/6980a91618101e92e8e5d5a61a910d817e08a5be/commonmark-0.9.1-py2.py3-none-any.whl"}
{"project_name": "selenium", "version": "3.141.0", "requires_python": null, "requires_dists": ["urllib3"], "location": "/home/jsirois/.pex/installed_wheels/4641f3d9c76dbc95a220098499db5f94929016bc/selenium-3.141.0-py2.py3-none-any.whl"}
{"project_name": "sentry-sdk", "version": "1.3.1", "requires_python": null, "requires_dists": ["urllib3>=1.10.0", "certifi", "aiohttp>=3.5; extra == \"aiohttp\"", "apache-beam>=2.12; extra == \"beam\"", "bottle>=0.12.13; extra == \"bottle\"", "celery>=3; extra == \"celery\"", "chalice>=1.16.0; extra == \"chalice\"", "django>=1.8; extra == \"django\"", "falcon>=1.4; extra == \"falcon\"", "flask>=0.11; extra == \"flask\"", "blinker>=1.1; extra == \"flask\"", "httpx>=0.16.0; extra == \"httpx\"", "pure-eval; extra == \"pure_eval\"", "executing; extra == \"pure_eval\"", "asttokens; extra == \"pure_eval\"", "pyspark>=2.4.4; extra == \"pyspark\"", "rq>=0.6; extra == \"rq\"", "sanic>=0.8; extra == \"sanic\"", "sqlalchemy>=1.2; extra == \"sqlalchemy\"", "tornado>=5; extra == \"tornado\""], "location": "/home/jsirois/.pex/installed_wheels/ab68a3cf5ac56c8c96554f44796eaa4f885c8060/sentry_sdk-1.3.1-py2.py3-none-any.whl"}
{"project_name": "shortuuid", "version": "1.0.1", "requires_python": ">=3.5", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/03baa4a717132c81ec12eb91fb570462fbd50507/shortuuid-1.0.1-py3-none-any.whl"}
{"project_name": "social-auth-app-django", "version": "5.0.0", "requires_python": ">=3.6", "requires_dists": ["social-auth-core>=4.1.0"], "location": "/home/jsirois/.pex/installed_wheels/11ca2f4826e34a4a0f57b94614f957f3395fe85a/social_auth_app_django-5.0.0-py3-none-any.whl"}
{"project_name": "social-auth-core", "version": "4.1.0", "requires_python": ">=3.6", "requires_dists": ["requests>=2.9.1", "oauthlib>=1.0.3", "requests-oauthlib>=0.6.1", "PyJWT>=2.0.0", "cryptography>=1.4", "defusedxml>=0.5.0rc1", "python3-openid>=3.0.10", "python-jose>=3.0.0; extra == \"all\"", "python3-saml>=1.2.1; extra == \"all\"", "cryptography>=2.1.1; extra == \"all\"", "python-jose>=3.0.0; extra == \"allpy3\"", "python3-saml>=1.2.1; extra == \"allpy3\"", "cryptography>=2.1.1; extra == \"allpy3\"", "cryptography>=2.1.1; extra == \"azuread\"", "python-jose>=3.0.0; extra == \"openidconnect\"", "python3-saml>=1.2.1; extra == \"saml\""], "location": "/home/jsirois/.pex/installed_wheels/473d2f869426ed095094494b66fd16cfc2beca6d/social_auth_core-4.1.0-py3-none-any.whl"}
{"project_name": "python3-openid", "version": "3.2.0", "requires_python": null, "requires_dists": ["defusedxml", "mysql-connector-python; extra == \"mysql\"", "psycopg2; extra == \"postgresql\""], "location": "/home/jsirois/.pex/installed_wheels/29f7ba2a382c5653ad16d3ca5a68eda366d1d3d0/python3_openid-3.2.0-py3-none-any.whl"}
{"project_name": "tomlkit", "version": "0.7.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": ["enum34<2.0,>=1.1; python_version >= \"2.7\" and python_version < \"2.8\"", "functools32<4.0.0,>=3.2.3; python_version >= \"2.7\" and python_version < \"2.8\"", "typing<4.0,>=3.6; python_version >= \"2.7\" and python_version < \"2.8\" or python_version >= \"3.4\" and python_version < \"3.5\""], "location": "/home/jsirois/.pex/installed_wheels/8a4ec8c767ff6166b394c4d0433c16d62cd37d0f/tomlkit-0.7.2-py2.py3-none-any.whl"}
{"project_name": "twine", "version": "3.4.2", "requires_python": ">=3.6", "requires_dists": ["pkginfo>=1.4.2", "readme-renderer>=21.0", "requests>=2.20", "requests-toolbelt!=0.9.0,>=0.8.0", "tqdm>=4.14", "importlib-metadata>=3.6", "keyring>=15.1", "rfc3986>=1.4.0", "colorama>=0.4.3"], "location": "/home/jsirois/.pex/installed_wheels/862f6326425ce3184f7f723557610bfe0ffe0cde/twine-3.4.2-py3-none-any.whl"}
{"project_name": "readme-renderer", "version": "29.0", "requires_python": null, "requires_dists": ["bleach>=2.1.0", "docutils>=0.13.1", "Pygments>=2.5.1", "six", "cmarkgfm<0.6.0,>=0.5.0; extra == \"md\""], "location": "/home/jsirois/.pex/installed_wheels/c295e822bad0162c4eb0dedd145f8558f43ff8cc/readme_renderer-29.0-py2.py3-none-any.whl"}
{"project_name": "bleach", "version": "4.1.0", "requires_python": ">=3.6", "requires_dists": ["packaging", "six>=1.9.0", "webencodings"], "location": "/home/jsirois/.pex/installed_wheels/b57b4e00b46618e15fbf3fb458bac0d9f1d8dd3c/bleach-4.1.0-py2.py3-none-any.whl"}
{"project_name": "webencodings", "version": "0.5.1", "requires_python": null, "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/137b54d665ff7c875d5b0e5a351c4dd2b77280bb/webencodings-0.5.1-py2.py3-none-any.whl"}
{"project_name": "docutils", "version": "0.17.1", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,>=2.7", "requires_dists": [], "location": "/home/jsirois/.pex/installed_wheels/e823d0fe0c83cbd1f9999132c792f65982a59c27/docutils-0.17.1-py2.py3-none-any.whl"}
{"project_name": "requests-toolbelt", "version": "0.9.1", "requires_python": null, "requires_dists": ["requests<3.0.0,>=2.0.1"], "location": "/home/jsirois/.pex/installed_wheels/157c19744397446fcde9b1f8785bc42706f4a5c1/requests_toolbelt-0.9.1-py2.py3-none-any.whl"}
{"project_name": "tqdm", "version": "4.62.2", "requires_python": "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,>=2.7", "requires_dists": ["colorama; platform_system == \"Windows\"", "py-make>=0.1.0; extra == \"dev\"", "twine; extra == \"dev\"", "wheel; extra == \"dev\"", "ipywidgets>=6; extra == \"notebook\"", "requests; extra == \"telegram\""], "location": "/home/jsirois/.pex/installed_wheels/eaf9cbb44c0983dd6ae40bb6e95b4624e2ed52e7/tqdm-4.62.2-py2.py3-none-any.whl"}
{"project_name": "importlib-metadata", "version": "4.8.1", "requires_python": ">=3.6", "requires_dists": ["zipp>=0.5", "typing-extensions>=3.6.4; python_version < \"3.8\"", "sphinx; extra == \"docs\"", "jaraco.packaging>=8.2; extra == \"docs\"", "rst.linker>=1.9; extra == \"docs\"", "ipython; extra == \"perf\"", "pytest>=4.6; extra == \"testing\"", "pytest-checkdocs>=2.4; extra == \"testing\"", "pytest-flake8; extra == \"testing\"", "pytest-cov; extra == \"testing\"", "pytest-enabler>=1.0.1; extra == \"testing\"", "packaging; extra == \"testing\"", "pep517; extra == \"testing\"", "pyfakefs; extra == \"testing\"", "flufl.flake8; extra == \"testing\"", "pytest-perf>=0.9.2; extra == \"testing\"", "pytest-black>=0.3.7; platform_python_implementation != \"PyPy\" and extra == \"testing\"", "pytest-mypy; platform_python_implementation != \"PyPy\" and extra == \"testing\"", "importlib-resources>=1.3; python_version < \"3.9\" and extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/67918f3ed71e2629b543eaf3b802aeea9e956234/importlib_metadata-4.8.1-py3-none-any.whl"}
{"project_name": "zipp", "version": "3.5.0", "requires_python": ">=3.6", "requires_dists": ["sphinx; extra == \"docs\"", "jaraco.packaging>=8.2; extra == \"docs\"", "rst.linker>=1.9; extra == \"docs\"", "pytest>=4.6; extra == \"testing\"", "pytest-checkdocs>=2.4; extra == \"testing\"", "pytest-flake8; extra == \"testing\"", "pytest-cov; extra == \"testing\"", "pytest-enabler>=1.0.1; extra == \"testing\"", "jaraco.itertools; extra == \"testing\"", "func-timeout; extra == \"testing\"", "pytest-black>=0.3.7; (platform_python_implementation != \"PyPy\" and python_version < \"3.10\") and extra == \"testing\"", "pytest-mypy; (platform_python_implementation != \"PyPy\" and python_version < \"3.10\") and extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/8e4e0651c85e30539e5b387b39284902cdfbc1a5/zipp-3.5.0-py3-none-any.whl"}
{"project_name": "keyring", "version": "23.1.0", "requires_python": ">=3.6", "requires_dists": ["importlib-metadata>=3.6", "SecretStorage>=3.2; sys_platform == \"linux\"", "jeepney>=0.4.2; sys_platform == \"linux\"", "pywin32-ctypes!=0.1.0,!=0.1.1; sys_platform == \"win32\"", "sphinx; extra == \"docs\"", "jaraco.packaging>=8.2; extra == \"docs\"", "rst.linker>=1.9; extra == \"docs\"", "pytest>=4.6; extra == \"testing\"", "pytest-checkdocs>=2.4; extra == \"testing\"", "pytest-flake8; extra == \"testing\"", "pytest-cov; extra == \"testing\"", "pytest-enabler>=1.0.1; extra == \"testing\"", "pytest-black>=0.3.7; platform_python_implementation != \"PyPy\" and extra == \"testing\"", "pytest-mypy; platform_python_implementation != \"PyPy\" and extra == \"testing\""], "location": "/home/jsirois/.pex/installed_wheels/eacdbd03218442b57ba3b2091a61c01221411118/keyring-23.1.0-py3-none-any.whl"}
{"project_name": "SecretStorage", "version": "3.3.1", "requires_python": ">=3.6", "requires_dists": ["cryptography>=2.0", "jeepney>=0.6"], "location": "/home/jsirois/.pex/installed_wheels/e3ffa860228b8eb0f94394650e48de656928f02d/SecretStorage-3.3.1-py3-none-any.whl"}
{"project_name": "jeepney", "version": "0.7.1", "requires_python": ">=3.6", "requires_dists": ["pytest; extra == \"test\"", "pytest-trio; extra == \"test\"", "pytest-asyncio; extra == \"test\"", "testpath; extra == \"test\"", "trio; extra == \"test\"", "async-timeout; extra == \"test\"", "trio; extra == \"trio\"", "async_generator; extra == \"trio\" and python_version == \"3.6\""], "location": "/home/jsirois/.pex/installed_wheels/e73e90d352c425d1bbf10c77230b87518d2f3c59/jeepney-0.7.1-py3-none-any.whl"}

real	0m1.229s
user	0m1.182s
sys	0m0.043s

That's the1 time write time for a graph - your approach would cache this one after building the repository pex. Then there is the read-walk time you'l have for each the N cases to determine their subset.

stuhood · 2021-09-02T03:34:38Z

Here's the worst case using my sample which is R=125 again:
...
So that's 2a + 2b for --spread since --spread does 2a ahead of time when it builds the repository PEX. Note also that this includes a real resolve and that takes most of the time.

Ok, thanks! According to pantsbuild/pants#12563 the time to digest-and-skip-capturing of mostly unchanged files should be pretty trivial (<200 ms), so it is very likely to be faster overall, even after fixing pantsbuild/pants#12688. I continue to be slightly weirded out by the "internal" composition of the --spread PEX (the PEX(-SPREAD)-INFO containing the list of reqs) vs external composition (an externally specified PEX_PATH containing the list of reqs), but it's hard to argue with the results.

Reviewing!

stuhood

Thanks a lot!

It seems like it would be great to be able to deprecate one of the modes (--unzip?) to remove some complexity... but this seems to have been able to slot in fairly easily regardless.

pex/pex_builder.py

The missing piece to get tools and --pex-repository working seamlessly was teaching PEXEnvironment about spreads since its the runtime piece that needs to know how to find and activate dists.

jsirois · 2021-09-02T15:08:06Z

The --unzip mode should simply become the default. The only reason it isn't today is that its a space hog, so I deliberately want to give the user the choice of tradeoffs. The --unzip mode for a PEX zipapp simply unzips the whole PEX under ~/.pex/unzipped_pexes and so the .bootstrap and all installed wheel chroots are duplicated there. The --spread mode, however, constructs ~/.pex/unzipped_pexes entries using symlinks for .bootstrap and all the unpacked wheel chroots that point into the single canonical location in ~/.pex/bootstraps/ and ~/.pex/installed_wheels respectively. I'll follow up to teach zipapp --unzip to work like spreading and then make the --unzip flag a noop / the default / deprecate.

jsirois · 2021-09-02T15:14:13Z

I continue to be slightly weirded out by the "internal" composition of the --spread PEX (the PEX(-SPREAD)-INFO containing the list of reqs) vs external composition (an externally specified PEX_PATH containing the list of reqs)

I don't understand this - I think it's a misconception. So, forgetting this change, the fundamental form of a PEX is loose. Thats a directory with a __main__.py and a PEX-INFO. That's what PEXBuilder builds and Pants v1 used internally because a loose PEX is ~3 times faster than a PEX zipapp in the noop case. Then a PEX zipapp is just that loose dir zipped up with a shebang on top. A spread is just a loose that has component parts zipped up. The PEX-SPREAD-INFO need not even exist, it just made the code cleaner. I could have complicated existing code in PEXEnvironment to detect the .deps/ were zips and not dirs and do the unpacking itself.

... Ok - there is the src/ dir I added to the spread form. That though is just to make a namespace so I can be free to add a top level pex -> __main__.py symlink as a convenience for running the PEX. That too is a distraction, the sources could all just be up at the top level like they are in a loose PEX and then the spread would just not contain a pex convenience entrypoint.

I'm thinking between the PEX-SPREAD-INFO keep the code clean hack and the src/ nesting to add a pex convenience script hack I confused things.

jsirois · 2021-09-02T15:17:20Z

And the official timings for the relevant bits of the discussion that got into "subsetting" in Pants terminology:

R=125

$ hyperfine -w2 -p "rm -rf big.pex big.pex.spread" "pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex -o big.pex" "pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex.spread -o big.pex.spread --spread"
Benchmark #1: pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex -o big.pex
  Time (mean ± σ):      7.630 s ±  0.058 s    [User: 7.069 s, System: 0.538 s]
  Range (min … max):    7.555 s …  7.756 s    10 runs
 
Benchmark #2: pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex.spread -o big.pex.spread --spread
  Time (mean ± σ):      1.402 s ±  0.037 s    [User: 1.242 s, System: 0.153 s]
  Range (min … max):    1.371 s …  1.481 s    10 runs
 
Summary
  'pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex.spread -o big.pex.spread --spread' ran
    5.44 ± 0.15 times faster than 'pex.with-spread.pex -r 3rdparty/python/requirements.txt --pex-repository repository.pex -o big.pex'

R=8

$ hyperfine -w2 -p "rm -rf pytest.pex pytest.pex.spread" "pex.with-spread.pex pytest --pex-repository repository.pex -o pytest.pex" "pex.with-spread.pex pytest --pex-repository repository.pex.spread -o pytest.pex.spread --spread"
Benchmark #1: pex.with-spread.pex pytest --pex-repository repository.pex -o pytest.pex
  Time (mean ± σ):     689.4 ms ±   5.3 ms    [User: 624.9 ms, System: 62.6 ms]
  Range (min … max):   680.2 ms … 699.8 ms    10 runs
 
Benchmark #2: pex.with-spread.pex pytest --pex-repository repository.pex.spread -o pytest.pex.spread --spread
  Time (mean ± σ):     385.1 ms ±   6.9 ms    [User: 335.4 ms, System: 48.7 ms]
  Range (min … max):   375.9 ms … 396.2 ms    10 runs
 
Summary
  'pex.with-spread.pex pytest --pex-repository repository.pex.spread -o pytest.pex.spread --spread' ran
    1.79 ± 0.03 times faster than 'pex.with-spread.pex pytest --pex-repository repository.pex -o pytest.pex'

R=1

$ hyperfine -w2 -p "rm -rf colors.pex colors.pex.spread" "pex.with-spread.pex ansicolors --pex-repository repository.pex -o colors.pex" "pex.with-spread.pex ansicolors --pex-repository repository.pex.spread -o colors.pex.spread --spread"
Benchmark #1: pex.with-spread.pex ansicolors --pex-repository repository.pex -o colors.pex
  Time (mean ± σ):     582.9 ms ±   8.6 ms    [User: 530.7 ms, System: 50.4 ms]
  Range (min … max):   572.9 ms … 602.5 ms    10 runs
 
Benchmark #2: pex.with-spread.pex ansicolors --pex-repository repository.pex.spread -o colors.pex.spread --spread
  Time (mean ± σ):     348.0 ms ±  12.5 ms    [User: 303.6 ms, System: 43.3 ms]
  Range (min … max):   337.6 ms … 371.8 ms    10 runs
 
Summary
  'pex.with-spread.pex ansicolors --pex-repository repository.pex.spread -o colors.pex.spread --spread' ran
    1.68 ± 0.07 times faster than 'pex.with-spread.pex ansicolors --pex-repository repository.pex -o colors.pex'

stuhood · 2021-09-02T19:19:15Z

I continue to be slightly weirded out by the "internal" composition of the --spread PEX (the PEX(-SPREAD)-INFO containing the list of reqs) vs external composition (an externally specified PEX_PATH containing the list of reqs)

I don't understand this - I think it's a misconception. So, forgetting this change, the fundamental form of a PEX is loose. Thats a directory with a __main__.py and a PEX-INFO. That's what PEXBuilder builds and Pants v1 used internally because a loose PEX is ~3 times faster than a PEX zipapp in the noop case. Then a PEX zipapp is just that loose dir zipped up with a shebang on top. A spread is just a loose that has component parts zipped up. The PEX-SPREAD-INFO need not even exist, it just made the code cleaner. I could have complicated existing code in PEXEnvironment to detect the .deps/ were zips and not dirs and do the unpacking itself.

At a fundamental level, aren't Spreads-for-installed-wheels basically eggs (from a consumption perspective)? Maybe in a slightly different layout? It seems perfectly natural (probably only because I'm used to the JVM and/or other compiled languages) to externally compose eggs (JAR/.a files) to build a workspace, but in reality that means making two independent trips to PEX: 1) do the resolve and produce a graph, 2) later, another call to produce a binary for a subset of the resolve.

I'm very used to build tools being the ones responsible for composing a bunch of compiler outputs into a binary (possibly by asking a linker to do so)... perhaps part of the reason this feels odd to me is that PEX is also an end-to-end build tool in some sense, and Pants can't really consume PEX piecemeal (although pex-tools starts to head in that direction?).

And the official timings for the relevant bits of the discussion that got into "subsetting" in Pants terminology:

Thanks! So it looks like for N=~90 and R=125, we'd be looking at 1.402 * 90 == 126.18 seconds in the worst case to actually build the subsets (the average will probably be about half of that). Building 125 transitive PEXes from a repository.pex took 3m19s (2m39s for intransitive PEXes) in my experiments... but that will actually also be significantly accelerated by --spread.

Maybe we'll end up using --spread ~everywhere internally, but still composing via the PEX_PATH? It occurred to me that we don't actually need to build a PEX to use the PEX_PATH, which would remove the 300ms I had mentioned earlier.

In any case: thanks!

jsirois · 2021-09-07T12:49:46Z

pex/pex_builder.py

+                    for f in self._chroot.filesets.get(fileset, ()):
+                        dest = os.path.join(work_dir, "src", f)
+                        safe_mkdir(os.path.dirname(dest))
+                        safe_copy(os.path.join(self._chroot.chroot, f), dest)


It turned out this line was a bug. The safe_copy tries to use hard linking, and when that succeeds, you can get a symlink out the other side if f went in as a symlink (hardlink of a symlink on linux is a symlink). Wrapping the LHS with an os.path.realpath fixes and lets --spread be used over in Pants where it nets a ~23x space savings vs main.

When running:

rm -rf ~/.cache/pants/lmdb_store ./pants fmt lint typecheck test build-support:: src:: tests::

I found:

du -sb lmdb_store.old lmdb_store.main lmdb_store.spread 1623506944 lmdb_store.old 1614954496 lmdb_store.main 1423679488 lmdb_store.spread

So that's:
old: 1623506944
PEX_PATH transitive: 1614954496 (-8552448) ~8MB saved.
spread: 1423679488 (-199827456) ~200MB saved, ~23x improvement.

IIUC the local lmdb savings should directly map to remote CAS savings / hit ratios.

It turned out this line was a bug. The safe_copy tries to use hard linking, and when that succeeds, you can get a symlink out the other side if f went in as a symlink (hardlink of a symlink on linux is a symlink). Wrapping the LHS with an os.path.realpath fixes and lets --spread be used over in Pants where it nets a ~23x space savings vs main.

It looks like sandbox outputs are currently symlink aware (because they don't use PosixFS::new_with_symlink_behavior(..., SymlinkBehavior:Oblivous) here), but that could be disabled if we think that it aligns with expected REAPI behavior? cc @tdyas

old: 1623506944
PEX_PATH transitive: 1614954496 (-8552448) ~8MB saved.
spread: 1423679488 (-199827456) ~200MB saved, ~23x improvement.

Nice. Note that the savings are very dependent on N: the total size of all transitive PEXes for the N~=95 case is ~244MB, vs >2.5GB before that change.

Did you measure performance while you were at it? Maybe too noisy in larger cases.

I did - perf was in the noise. All runs were 17 minutes +/-.

I did - perf was in the noise. All runs were 17 minutes +/-.

Got it. Yea, I think that that is likely to dependent on N as well (particularly for the pre-PEX_PATH change). Subset building adds up when you do enough of it.

jsirois requested review from Eric-Arellano and stuhood August 31, 2021 01:50

Fix attr vendoring.

8299202

jsirois mentioned this pull request Aug 31, 2021

Fix PyPy integration tests in CI. #1432

Merged

jsirois added 2 commits August 31, 2021 09:35

Remove un-needed loose bootstrap pre-unpack.

5ad4b73

This had no noticeable impact on performance.

Remove attr use in spread.py.

e34fcf5

This added ~60 ms to bootstrap with ~30ms going to import time and ~30ms going to attr dynamics.

Merge remote-tracking branch 'refs/remotes/upstream/main' into issues…

d0ad3ef

…/1424

jsirois added 2 commits August 31, 2021 13:11

Merge remote-tracking branch 'refs/remotes/upstream/main' into issues…

3d76f34

…/1424

Actually use __PEX_EXE__ redirection for spreads.

8c08ec0

stuhood mentioned this pull request Sep 1, 2021

Improve descriptions of the process executions for PEX_PATH composition. pantsbuild/pants#12736

Merged

stuhood approved these changes Sep 2, 2021

View reviewed changes

pex/pex_builder.py Show resolved Hide resolved

Make runtime Pex fully robust to spreads.

3185188

The missing piece to get tools and --pex-repository working seamlessly was teaching PEXEnvironment about spreads since its the runtime piece that needs to know how to find and activate dists.

jsirois merged commit cbe7b53 into pex-tool:main Sep 2, 2021

jsirois deleted the issues/1424 branch September 2, 2021 15:33

jsirois commented Sep 7, 2021

View reviewed changes

jsirois mentioned this pull request Jul 26, 2023

consume packed wheel cache in zipapp creation #2175

Closed

jsirois mentioned this pull request Nov 16, 2023

Pex zip-creation takes a very long time for torch>=2 #2292

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce the spread app layout. #1431

Introduce the spread app layout. #1431

jsirois commented Aug 31, 2021 •

edited

Loading

jsirois commented Aug 31, 2021

jsirois commented Aug 31, 2021

jsirois commented Aug 31, 2021 •

edited

Loading

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 •

edited

Loading

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 •

edited

Loading

jsirois commented Sep 1, 2021 •

edited

Loading

stuhood commented Sep 1, 2021

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 •

edited

Loading

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021 •

edited

Loading

jsirois commented Sep 2, 2021 •

edited

Loading

stuhood commented Sep 2, 2021 •

edited

Loading

stuhood left a comment

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021 •

edited

Loading

jsirois commented Sep 2, 2021

stuhood commented Sep 2, 2021

jsirois Sep 7, 2021 •

edited

Loading

stuhood Sep 7, 2021 •

edited

Loading

jsirois Sep 7, 2021

stuhood Sep 7, 2021 •

edited

Loading

Introduce the spread app layout. #1431

Introduce the spread app layout. #1431

Conversation

jsirois commented Aug 31, 2021 • edited Loading

jsirois commented Aug 31, 2021

jsirois commented Aug 31, 2021

jsirois commented Aug 31, 2021 • edited Loading

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 • edited Loading

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 • edited Loading

jsirois commented Sep 1, 2021 • edited Loading

stuhood commented Sep 1, 2021

jsirois commented Sep 1, 2021

stuhood commented Sep 1, 2021 • edited Loading

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021 • edited Loading

jsirois commented Sep 2, 2021 • edited Loading

stuhood commented Sep 2, 2021 • edited Loading

stuhood left a comment

Choose a reason for hiding this comment

jsirois commented Sep 2, 2021

jsirois commented Sep 2, 2021 • edited Loading

jsirois commented Sep 2, 2021

stuhood commented Sep 2, 2021

jsirois Sep 7, 2021 • edited Loading

Choose a reason for hiding this comment

stuhood Sep 7, 2021 • edited Loading

Choose a reason for hiding this comment

jsirois Sep 7, 2021

Choose a reason for hiding this comment

stuhood Sep 7, 2021 • edited Loading

Choose a reason for hiding this comment

jsirois commented Aug 31, 2021 •

edited

Loading

jsirois commented Aug 31, 2021 •

edited

Loading

stuhood commented Sep 1, 2021 •

edited

Loading

stuhood commented Sep 1, 2021 •

edited

Loading

jsirois commented Sep 1, 2021 •

edited

Loading

stuhood commented Sep 1, 2021 •

edited

Loading

jsirois commented Sep 2, 2021 •

edited

Loading

jsirois commented Sep 2, 2021 •

edited

Loading

stuhood commented Sep 2, 2021 •

edited

Loading

jsirois commented Sep 2, 2021 •

edited

Loading

jsirois Sep 7, 2021 •

edited

Loading

stuhood Sep 7, 2021 •

edited

Loading

stuhood Sep 7, 2021 •

edited

Loading