Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Better parameter learning and inductive types #191

Merged
merged 232 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
232 commits
Select commit Hold shift + click to select a range
627a833
mnist wip
rtjoa Sep 19, 2023
d33207e
[Autodiff] Support matrix vars, sin, cos
rtjoa Sep 22, 2023
0657b89
Add MNIST
rtjoa Sep 29, 2023
bb79ba7
Remove old mnist
rtjoa Sep 29, 2023
c0226ce
[Autodiff] Arbitrary pr-dependent loss fns
rtjoa Dec 14, 2023
9285201
Add KL-divergence loss
rtjoa Dec 14, 2023
4b9e7e2
Example: approximating int dists
rtjoa Dec 14, 2023
119bea4
tweak lcd
rtjoa Dec 19, 2023
127d373
lcd test
rtjoa Dec 19, 2023
79db3e0
split adnode.jl out of core.jl
rtjoa Dec 19, 2023
64f8991
add LogPr
rtjoa Dec 22, 2023
421aca0
arbitrary interleaving
rtjoa Dec 23, 2023
dc1e647
remove nodelogpr
rtjoa Dec 23, 2023
c7fcb10
remove commented adnodes: div, exp
rtjoa Dec 23, 2023
d1dd9e3
remove Variable
rtjoa Dec 23, 2023
2b44dd1
shrink AD interface
rtjoa Dec 23, 2023
0b99903
remove learned_cheap_dists example
rtjoa Dec 23, 2023
cf1fc3b
remove mnist example
rtjoa Dec 23, 2023
d9daee6
fix darts example
rtjoa Dec 24, 2023
ee12203
fix qc
rtjoa Dec 24, 2023
bce589b
only expand once in train_pr
rtjoa Dec 24, 2023
b44d5c2
Revert "remove nodelogpr"
rtjoa Dec 24, 2023
ef2f8dd
update stlc version and reset init size to 5
rtjoa Dec 24, 2023
d5235a9
add pr_mixed, support_mixed, and dir for autodiff_pr
rtjoa Dec 24, 2023
ff69329
have with_concrete_ad_flips support LogPr
rtjoa Dec 24, 2023
7cd0563
add neg_entropy
rtjoa Dec 27, 2023
ae19e01
train generator w entropy
rtjoa Dec 27, 2023
096f0b8
approx entropy
rtjoa Jan 13, 2024
96104d6
show loss
rtjoa Jan 13, 2024
43099e8
add entropy approx output
rtjoa Jan 14, 2024
c3460b4
refactor benchmark infra
rtjoa Jan 15, 2024
575c8c0
qc benchmarks small changes
rtjoa Jan 15, 2024
3e88e1e
add BST benchmark
rtjoa Jan 15, 2024
0aa5a2c
update demo_bst for qc7.5
rtjoa Jan 16, 2024
b46ff35
qc bench: add mixed loss, always save app dist
rtjoa Jan 16, 2024
ee639c2
remove rest of qc outupt
rtjoa Jan 16, 2024
6940520
undo: logpr, expand_logpr of const
rtjoa Jan 16, 2024
27de651
[qc] add few apps loss, print var vals, preexisting var val warning
rtjoa Jan 21, 2024
ba66dde
Merge branch 'qc7.5-approx-entropy' of https://github.com/Juice-jl/Di…
rtjoa Jan 21, 2024
e8bc4f3
add qc stlc size 5 formatter
rtjoa Jan 22, 2024
1d94939
Add to_graph
rtjoa Jan 31, 2024
290010b
graphviz loss adnode graph
rtjoa Jan 31, 2024
eccd81c
Update examples or mark as deprecated
rtjoa Jan 31, 2024
d03a1bd
Update QC README
rtjoa Jan 31, 2024
76edb70
refactor qc runner to use lossmgrs
rtjoa Feb 3, 2024
688ce36
Add Plots to Project.toml
rtjoa Feb 4, 2024
bf16b1b
Add Infiltrator to Project.toml
rtjoa Feb 4, 2024
c4d0657
Generate learning curve image
rtjoa Feb 4, 2024
63a3be5
qc bench: print more times
rtjoa Feb 4, 2024
8c012c2
add sample_as_dist
rtjoa Feb 4, 2024
3d282b4
Add STLC sampling entropy loss
rtjoa Feb 4, 2024
69dcb2f
fix nodelogpr bug
rtjoa Feb 4, 2024
8492891
impl frombits_as_dist for inductive and enum
rtjoa Feb 4, 2024
ee3d6a8
qc bench: add sampling stlc term entropy
rtjoa Feb 4, 2024
e55d38b
qc bench: add sampling bst term entropy loss
rtjoa Feb 4, 2024
9411e3b
qc bench: plots in headless, bst with dummy values
rtjoa Feb 4, 2024
bd89aed
qc bench: print commit
rtjoa Feb 4, 2024
3852cdc
qc bench: svg learning rate, fix sampling entropy loss, avoid overwrite
rtjoa Feb 4, 2024
238b2f7
add stop if inf or nan in train\!, old unif impl
rtjoa Feb 5, 2024
27f5391
linear instead of sqrt LR adjustment
rtjoa Feb 5, 2024
7634f27
draft stlc type-based generator
rtjoa Feb 12, 2024
3b2ab88
add stlc ae
rtjoa Feb 21, 2024
0a92e7a
fix stlc format
rtjoa Feb 21, 2024
532cca0
format bespoke stlc
rtjoa Feb 21, 2024
f0ec412
add stlc type-based generator
rtjoa Feb 21, 2024
5fe9ece
add qcbench.sh
rtjoa Feb 21, 2024
7ebcbcd
add unif_half2
rtjoa Feb 21, 2024
21e6d42
add unif3, unif2
rtjoa Feb 28, 2024
e934f72
add type-based bst generator
rtjoa Feb 28, 2024
6589d05
add tree size and output
rtjoa Feb 29, 2024
9ef8e0a
RBT draft, gen bespoke stlc coq, outputs
rtjoa Feb 29, 2024
48ebf82
add TB RBT
rtjoa Mar 4, 2024
e7ad436
RBT: learn_leaf_weights and fix bugs
rtjoa Mar 4, 2024
23875ea
rbt fix name
rtjoa Mar 4, 2024
dfd5af4
add sampling entropy rbt loss, loop
rtjoa Mar 7, 2024
e72ffe7
generalize train, two curves
rtjoa Mar 9, 2024
8daf00f
allow RBT to depend on parent flip
rtjoa Mar 9, 2024
d422f57
have sample take rng
rtjoa Mar 13, 2024
c3b4220
runstate and fix sampling entropy sign
rtjoa Mar 13, 2024
9f4fc53
run_benchmark refactor
rtjoa Mar 13, 2024
a4b3952
fix stlc gen
rtjoa Mar 13, 2024
33ddf6d
add threebools
rtjoa Mar 14, 2024
b22c0f4
parallel training runs
rtjoa Mar 14, 2024
67103ff
reduce code duplication in train!
rtjoa Mar 14, 2024
1981e7e
vec/tuple: add frombits_as_dist and prob_equals
rtjoa Mar 14, 2024
62e23f4
add threebools
rtjoa Mar 14, 2024
2cbb6d5
debugging weird behavior wip
rtjoa Mar 14, 2024
9ddfb26
exact entropy flipflipflip
rtjoa Mar 15, 2024
e701e92
single flip sampling entropy test
rtjoa Mar 15, 2024
c3f4830
gumbel and other junk!
rtjoa Mar 16, 2024
5129d53
Revert "gumbel and other junk!"
rtjoa Mar 25, 2024
a026b3b
reinforce
rtjoa Mar 26, 2024
9c96ec4
tweaks
rtjoa Apr 1, 2024
82dd5aa
Add comment about roots/GC to BDDCompiler
rtjoa Apr 1, 2024
defe37f
LRUset
rtjoa Apr 1, 2024
82bc2b1
save actual loss curves
rtjoa Apr 2, 2024
f090b98
@inductive wip
rtjoa Apr 3, 2024
c08b002
add typevars to front of inductive ctor params
rtjoa Apr 4, 2024
6cf7898
finish @inductive
rtjoa Apr 4, 2024
2facb51
overhaul @inductive
rtjoa Apr 5, 2024
3596646
update @inductive users
rtjoa Apr 5, 2024
f39c49e
conditional entropy, @match
rtjoa Apr 7, 2024
7a9c891
conditional reinforce entropy loss
rtjoa Apr 7, 2024
0695448
IFC wip
rtjoa Apr 7, 2024
5a96a5d
small ifc progress
rtjoa Apr 16, 2024
e72e74f
entropy: add failure penalty, log pct meeting; rbt: fix to_coq
rtjoa Apr 16, 2024
fdaa40f
rbt to coq size
rtjoa Apr 21, 2024
3de8fb9
remove failure_penalty, take more samples
rtjoa Apr 21, 2024
b9d2ad6
update tag
rtjoa Apr 21, 2024
7ada1d2
add stats.py
rtjoa Apr 22, 2024
6b1a96f
update stats
rtjoa Apr 22, 2024
4838fe9
add additive failure penalty
rtjoa Apr 22, 2024
42bb898
add order inv, less dummy nums in generator, and entropy ignore nums
rtjoa Apr 22, 2024
f0d0fbe
oops unswap ignore_nums
rtjoa Apr 22, 2024
74ab57f
move and update stats.py
rtjoa Apr 22, 2024
ca26ead
improvements
rtjoa Apr 24, 2024
6875342
add support_as_dst
rtjoa Apr 25, 2024
28c44da
qc: flip_for with arbitrary dependents
rtjoa Apr 25, 2024
40f3eb2
rbt intwidth, dependents
rtjoa Apr 25, 2024
8a8e9df
add as_dist to pr, use pr for support
rtjoa Apr 26, 2024
de2941a
rbt longer training
rtjoa Apr 26, 2024
3a2f507
rbt unif depth
rtjoa Apr 26, 2024
585a125
add Opt.bind, Opt.map
rtjoa May 1, 2024
2c411c7
typebased stlc, tocoq, dependents
rtjoa May 1, 2024
64d2c33
main: tb stlc
rtjoa May 1, 2024
339fd25
tb bst, tocoq, dependents
rtjoa May 1, 2024
01ab19e
stlc and bst: define gSized in to_coq
rtjoa May 2, 2024
a820d28
bst and stlc stats
rtjoa May 2, 2024
48d7f90
cond ent takes eq, fix tbbst tocoq
rtjoa May 6, 2024
0df1b38
learn stlc var nums
rtjoa May 7, 2024
c593388
stacktail stlc
rtjoa May 9, 2024
96f4f02
rbt stack
rtjoa May 9, 2024
7746953
stats: cformat, print path
rtjoa May 10, 2024
7613dc0
gitignore stats
rtjoa May 10, 2024
43eb535
stlc eq structure
rtjoa May 10, 2024
d0083ba
add uint max/min
rtjoa May 13, 2024
4353007
inductive: add variants
rtjoa May 13, 2024
5d00e27
sat_num_apps has app and wip derive
rtjoa May 13, 2024
e5d3d49
add forgiveness
rtjoa May 13, 2024
21e59b5
actual loss here only consider some samples when rand_forgiveness for…
rtjoa May 13, 2024
100c19b
fix typo
rtjoa May 13, 2024
7e4b0b5
variant keys actual function
rtjoa May 14, 2024
a5fe094
derived to_coq progress, refactor forgiveness
rtjoa May 14, 2024
40faa9a
uncomment
rtjoa May 14, 2024
e0d8bee
derived stlc seems to work
rtjoa May 14, 2024
c927f70
mess with spacing
rtjoa May 14, 2024
66d2428
a bit more messing with spacing
rtjoa May 14, 2024
9ff9998
derive progress
rtjoa May 17, 2024
54790f2
derived coq typechecks for empyt stack
rtjoa May 17, 2024
abf7016
small tweaks before annoying extraction change
rtjoa May 17, 2024
614711f
hundredths and fix tuple extraction
rtjoa May 17, 2024
5d0f0ea
main
rtjoa May 17, 2024
b70653b
Revert "main"
rtjoa May 17, 2024
beb3a87
derived RBT
rtjoa May 17, 2024
315bc46
might type but its too forgiving
rtjoa May 19, 2024
d183e72
fix might_typecheck bug
rtjoa May 19, 2024
6a9f980
allow tobits for empty tuple
rtjoa May 19, 2024
a2d9a0c
stricter might_typecheck (check var nums)
rtjoa May 19, 2024
3b94005
stlc var numbers
rtjoa May 20, 2024
a45721b
print mle metric
rtjoa May 22, 2024
3d32996
STLC samples
rtjoa May 23, 2024
31db1c3
update main and tool
rtjoa May 24, 2024
33bdcc2
fix syntax error
rtjoa May 24, 2024
1d57bf6
lang wip
rtjoa May 26, 2024
559da22
Dice add inductivetype
rtjoa May 27, 2024
f3431cc
bespoke stlc arbitrary in lang
rtjoa May 27, 2024
05b2ffa
match on int using O/S
rtjoa May 27, 2024
6b7f4c2
interp L
rtjoa May 27, 2024
306cafa
derive lang
rtjoa May 28, 2024
22a5879
fix derive lang
rtjoa May 28, 2024
e0713ea
lang derived stlc coq compiles
rtjoa May 28, 2024
1b0e6c8
add combinatorics
rtjoa May 28, 2024
85f0032
get langderivedgenerator to work for BST
rtjoa May 28, 2024
9f0e22d
fix inductive for matches on (types with only one ctor? ctors with no…
rtjoa May 29, 2024
dca7b1c
derive siblings compiles for rbt
rtjoa May 29, 2024
5b20336
lang derived siblings stlc compiles
rtjoa May 29, 2024
f71e168
add stlcmaytype
rtjoa May 29, 2024
836d31d
LD: add arbitrary_prims
rtjoa Jun 3, 2024
3ed1f06
small tweaks
rtjoa Jun 4, 2024
50bd5f1
update stats
rtjoa Jun 4, 2024
e1b1533
better tool
rtjoa Jun 5, 2024
414f5ad
lang bst
rtjoa Jun 6, 2024
6e47222
update main
rtjoa Jun 6, 2024
39f4146
update main
rtjoa Jun 6, 2024
0a9c102
main for rbt 1k
rtjoa Jun 6, 2024
4f03b63
main for rbt 1k
rtjoa Jun 6, 2024
b342394
stlc thin
rtjoa Jun 7, 2024
a70bd5a
no dtt
rtjoa Jun 7, 2024
e180e52
add bound
rtjoa Jun 10, 2024
22cea70
RBT bound
rtjoa Jun 12, 2024
9fd48b5
switch workload
rtjoa Jun 12, 2024
79389b8
ace bound
rtjoa Jun 14, 2024
bc54e01
bst bound
rtjoa Jun 14, 2024
3f2ce62
stlc well
rtjoa Jun 14, 2024
a8a90a0
rbt bigger
rtjoa Jun 15, 2024
96c37eb
stlc faster
rtjoa Jun 15, 2024
ebf0330
x
rtjoa Jun 15, 2024
3aa4faa
update renamer
rtjoa Jun 16, 2024
cbbd700
rbt small
rtjoa Jun 17, 2024
57bfd1a
bst small
rtjoa Jun 17, 2024
95b41af
small fix
rtjoa Jun 17, 2024
fceceb1
bespoke se
rtjoa Jun 22, 2024
e611329
more lrs
rtjoa Jun 22, 2024
e6177e7
repro and space2
rtjoa Jun 22, 2024
057b790
eval repo scripts
rtjoa Jun 22, 2024
8239f53
stlc bespoke 5 1
rtjoa Jun 26, 2024
544cad2
update tag
rtjoa Jun 26, 2024
76bebd9
add faaster
rtjoa Jun 26, 2024
07bf4ea
51 se
rtjoa Jun 29, 2024
583a5b8
se force meets
rtjoa Jul 7, 2024
44bd0d3
structure
rtjoa Jul 8, 2024
f0b35b5
ex unif stlc
rtjoa Jul 8, 2024
0d63501
idk
rtjoa Jul 10, 2024
b4bfa4c
rbt ablation
rtjoa Jul 10, 2024
9a181cc
fixes
rtjoa Jul 10, 2024
e6ef8eb
simpler ace
rtjoa Jul 10, 2024
630d819
fix
rtjoa Jul 10, 2024
dcebc81
no samples
rtjoa Jul 10, 2024
c4aa247
rbt
rtjoa Jul 10, 2024
1d1cf39
rbt for unif depth
rtjoa Jul 10, 2024
6b79481
rbt speconly
rtjoa Jul 10, 2024
c637453
after deadine
rtjoa Jul 14, 2024
78a5d2c
attempt faster stlc
rtjoa Jul 16, 2024
d2f1ab6
fix typo
rtjoa Jul 16, 2024
b6432be
bools fig
rtjoa Jul 18, 2024
912c558
tikz plot
rtjoa Jul 18, 2024
19293c5
remove codecov from CI
rtjoa Jul 18, 2024
8c8869b
move tours to tests
rtjoa Jul 19, 2024
62bbd1c
remove packages
rtjoa Jul 19, 2024
e2df3c0
remove qc folder
rtjoa Jul 19, 2024
3bdd389
remove stats and other files
rtjoa Jul 19, 2024
38da216
remove mkeval.sh
rtjoa Jul 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ jobs:
run:
julia --project --check-bounds=yes --depwarn=yes -e 'import Pkg; Pkg.test(; coverage=true)'

- name: Codecov Upload
run:
julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder());'
# - name: Codecov Upload
# run:
# julia -e 'using Pkg; Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder());'
11 changes: 6 additions & 5 deletions examples/darts.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,22 @@ all(itr) = reduce(&, itr)
any(itr) = reduce(|, itr)

DARTS_PER_COLOR = [1, 2, 10] # number of red, green, and blue darts
weights = [var!("r", 1), var!("g", 1), var!("b", 1)]
weights = [Var("r"), Var("g"), Var("b")]
var_vals = Valuation(weight => 1 for weight in weights)

all_colors_receive_own_dart = all(
any(flip(weight / sum(weights)) for _ in 1:num_own_darts)
for (num_own_darts, weight) in zip(DARTS_PER_COLOR, weights)
)

pr(all_colors_receive_own_dart) # 0.182
train_vars!([all_colors_receive_own_dart]; epochs=1000, learning_rate=0.3)
pr_mixed(var_vals)(all_colors_receive_own_dart) # 0.182
train!(var_vals, -LogPr(all_colors_receive_own_dart); epochs=1000, learning_rate=0.3)

# We've increased the chance of success!
pr(all_colors_receive_own_dart) # 0.234
pr_mixed(var_vals)(pr_all_colors_receive_own_dart) # 0.234

# Compute what ratio we actually need to paint the target:
[compute(weight/sum(weights)) for weight in weights]
[compute(var_vals, weight/sum(weights)) for weight in weights]
# 3-element Vector{Float64}:
# 0.46536681058883267
# 0.3623861813855715
Expand Down
19 changes: 0 additions & 19 deletions examples/qc/README.md

This file was deleted.

105 changes: 0 additions & 105 deletions examples/qc/demo_bst.jl

This file was deleted.

68 changes: 0 additions & 68 deletions examples/qc/demo_natlist.jl

This file was deleted.

113 changes: 0 additions & 113 deletions examples/qc/demo_sortednatlist.jl

This file was deleted.

Loading
Loading