Demo symbolic-pymc on radon example in pymc4

[twiecki]

This model is non-centered: https://github.com/pymc-devs/pymc4/blob/7b144fa5be8b3b3f7e5ea9e9f3222988ab60a87f/notebooks/radon_hierarchical.ipynb

Would be great to see symbolic-pymc applied on this example to sample it in centered way.

[brandonwillard]

Looks like we need to update the TF and TFP dependencies in order to run that PyMC4 code alongside symbolic-pymc. Just put in a PR for that: #68.

[brandonwillard]

I've sketched out the boilerplate work for doing this in PyMC4 here. Looks like we only need to specify the graph manipulation in log space from here.

[twiecki]

This is super useful (and very cool). Why can't the transform we have already implemented for the PyMC3 example be re-used here?

Also, it highlights that we need a mechanism to switch out the logp function of a model object.

[brandonwillard]

I've updated it to include an actual transform. It's probably not the exact one to finish the job, but it shows everything in action.

[brandonwillard]

FYI: This makes for a great example of how sample space graphs are much more efficient for certain optimizations/transformations/reformulations/etc.

I'll add an explanation as I keep updating the gist for this issue.

[brandonwillard]

The Gist has been updated to reflect a working "measure-space" (i.e. log-likelihood) transformation implemented in symbolic-pymc. It's not exactly the same transformation as our original "sample-space" transformation in PyMC3, since it's missing some algebraic identities that would make it produce exactly the same log-likelihood.

Nevertheless, it demonstrates all the mechanics needed to do real work with symbolic-pymc on a non-trivial log-likelihood and it also appear to improve sampling a little bit.

Before (from the PyMC4 Notebook)

After

[brandonwillard]

I've moved this example to a more readable format here. The write-up still needs a bit of work (e.g. a better initial sampling point, miniKanren goal explanations), and the transform itself needs to use the full form of a normal log-likelihood to "correctly" handle the scale part of the rewriting (i.e. removal of the additive log(sigma) term).