filtering a subset of parameters

[lihebi]

Hi, a related question, is it possible to filter a subset of parameters? The use case is to apply L2 decay on weights but not bias.

[lihebi]

Sorry, miss-click

[MikeInnes]

You should be able to use filter as normal if you just want to filter matrices vs. vectors. It'd be more robust if you just write a function to pull the weights out of dense layers explicitly, though.

[lihebi]

Thanks for the reply. This relates to #160, and I accidentally hit Ctrl-Enter way before I wanted.

The params returned by Flux.params() does not seem to contain information about which layers they come from. I managed to pull weights from dense and conv layers:

function weight_params(m::Chain, ps=Flux.Params())
    map((l)->weight_params(l, ps), m.layers)
    ps
end
weight_params(m::Dense, ps=Flux.Params()) = push!(ps, m.W)
weight_params(m::Conv, ps=Flux.Params()) = push!(ps, m.weight)
weight_params(m::ConvTranspose, ps=Flux.Params()) = push!(ps, m.weight)
weight_params(m, ps=Flux.Params()) = ps