Juice v0.4 refactor

[khosravipasha]

• Param Learning

  • EM
  • EM CPU
  • Regurlarization (Softness)
  • SGD version. Best option seems to be to customize rrules from ChainRules.jl. Then can grad/optimize easily with Zygote or Flux.

• Queries/Transformation for the new BitCircuit type. CPU and CUDA versions.

  • EVI
  • MAR
  • MAP (approx)
  • Sample
  • Conditionals
  • All marginals
  • Marginal MAP algorithm. Would need to rethink the algorithm a bit since now have any input types.

• Structures

  • Splits, and clones
  • Learn structures from missing data
  • HCLT
  • Strudel
  • RAT-SPN with option to specify leaf type
  • PD Structures for images?
  • Other common structures (fully factorized, NB, HMM, markov chains, etc)

• Tests

  • Tests for EM learning with missing in the inputs
  • Test with different input types for likelihood, flow, MAP, Sample, etc
    • Binomial
    • Categorical
    • Indicator
  • Better small circuits for test cases, little_4var might be too simple.
  • Test for conditional sampling of circuits

• Docs

  • GPU Docs in manual
  • Notebook for learning
  • This seems nice for adding citations if needed https://github.com/ali-ramadhan/DocumenterCitations.jl

• Misc

  • Allow Multiple InputNode type for CUDA Kernels #120
  • Float input types had some issues, need to double check
  • Lazy Batching (batchview??) not sure if worth it https://mldatapatternjl.readthedocs.io/en/latest/documentation/dataview.html#as-vector-of-batches

• IO

  • DensityEstimationData.jl
  • JuiceModelZoo.jl
  • Read/Write of circuits text format.
    • Faster Version
    • Support direct read/write to zipped format gzip, etc
  • Binary format for circuit for more efficient saving
  • Zoo related IO; zoo_psdd, zoo_pc, etc.
  • TikZ plotting now broken, need slight modification