Proposal: Move params to Zygote

[cortner]

I'm just starting to look into the Flux eco-systems, primarily for Zygote rather than Flux itself and I was struck that Params lives in Zygote while params lives in Flux. This form of differentiation

Zygote.gradient( () -> loss(...),  params(model) )

is very convenient and elegant and useful for any nested structure, not just those built from Zygote.

Is there any chance to simply move this functionality into Zygote so that it can be used without depending on Flux? If I understand it correctly that would not even break backward compatibility.

[ToucheSir]

My inclination is no, but for a good reason. The nested structural traversal params does relies on https://github.com/FluxML/Functors.jl. You'll note that this is not a dependency of Zygote, and that's because it's quite a bit more opinionated than we want a generic AD library to be.

Another reason to not move params into Zygote is because we're trying to move away from using implicit params (that is, Params) altogether. #1864 will add the ability to take gradients with respect to an entire model structure, no calls to params required :)

[cortner]

That’s a good point - and I missed that. Ok then I need to take a look at your new strategy. Very curious. Thanks for the quick response.

[cortner]

Where can I find the discussion about moving away from implicit parameters? That PR seems to be a really small aspect?

[ToucheSir]

Unfortunately there isn't a real centralized source for the whole thing, but reading through the issues in Explicit Parameter Transition should give you a decent primer. The best (though somewhat out of date) description of the effort may be found in #637. You may also be interested in a look through the Optimisers.jl docs.