Add --update-all option and update all dependencies with their hashes?

[jotes]

Hey @peterbe!
I work with a lot of legacy projects that don't add hashes to their dependencies and I'd like to improve their security.
What do you think about an option which would just scan through all entries and add hashes to them?
Currently, I'm working around this with shell scripts but maybe it's a good idea to include it here.

I can even implement it if you think it will be useful.

Thanks in advance, hashin is awesome!

[peterbe]

What would be totally neat! When you do hashin Django lxml foobar and you already have those packages in the requirements.txt it will basically do an upgrade.

So you could basically do something like this:

if not args.packages and args.update_all:
  args.packages = get_all_current_package_names(args.file or 'requirements.txt')

(I don't know if mutating the Argparse object is a good idea, but the idea is there).

I look forward to a pull request. If you struggle to test it or how to format anything, let me know and we can complete it together.

[peterbe]

Another option would be to make it interactive. E.g. hashin -i --update-all -r requirements/default.txt. Then it could show an interactive prompt for each and every possible upgrade. E.g.

You have Django==2.0.4 installed, Django==2.0.5 is available. Upgrade? [Y/n]

How does yarn do interactive upgrades?

[jotes]

Thanks @peterbe, I'll reach out you in the next week.

[peterbe]

Any luck with this one?

[peterbe]

hashin doesn't have a dry-run functionality because it's never really been necessary. However, for a functionality like this it certainly would be useful.

[jotes]

hey @peterbe, I was totally busy last week and couldn't look into this. However, I'm free tomorrow and this issue is on the top of the list.

[peterbe]

Hi @jotes Check out what just landed! #96

Can help me test it? I haven't made a release yet.

[jotes]

@peterbe YUP! I'll ping you in 10-12h with the results of QA.
I've realized that you've basically made this whole feature... Thanks and I apologize for so long time with no response.
HUGE kudos :)

[jotes]

@peterbe hey,
I've ran hashin on my current project :)

I'm not sure if that's a bug, but I've noticed missing hashes in some of dependencies.

For easier debug you can take a look at these files:
requirements-before-hashin.txt
https://gist.github.com/jotes/c4942951bd09d6a4153815508446d382
requirements-after-hashin.txt
https://gist.github.com/jotes/b73829d018117ec2531072cae31a47a5

I hope this bug is reproducible. Feel free to ask if you need anything :-)

Thanks again!

[peterbe]

@jotes Thanks! It's not really a bug in the code. It's just a confusion of expectations. You have a requirements file that did not already have hashes. Generally, this is not something hashin is used to deal with. Independent of the new --update-all --interactive feature, what would you expect if you run:

$ hashin -r requirements-before-hashin.txt alembic

I think the output of that would become something like this: https://gist.github.com/peterbe/bb003c7bfbabfeaa682027234176b2fe

Then, if you try to use that file with pip install you'd get this error:

$ pip install -r requirements-before-hashin.txt
...

Hashes are required in --require-hashes mode, but they are missing from some requirements. Here is a list of those requirements along with the hashes their downloaded archives actually had. Add lines like these to your requirements files to prevent tampering. (If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash.)
    amqp==2.2.2 --hash=sha256:4e28d3ea61a64ae61830000c909662cb053642efddbe96503db0e7783a6ee85b
    anyjson==0.3.3 --hash=sha256:37812d863c9ad3e35c0734c42e0bf0320ce8c3bed82cd20ad54cb34d158157ba
    behave==1.2.5
...

[peterbe]

In short, I don't think that's a bug in the new feature of the context of this issue. However, it's an interesting thing that will/can happen. How should hashin deal with it?

Perhaps it could analyze the requirements file and do something like:

$ hashin -r my-requirements.txt Django celery requests
Traceback (most recent call last):
...
Exception: Requirements file does not have hashes for all packages.

[peterbe]

What do you think?

[jotes]

@peterbe You're right, thanks for the clarification :-) I think that an explicit exception is totally okay.