GitHub - cats256/ML-experiments

This repo stores various experiments I have done over time. More notebooks will be uploaded over time as I clean and organize things up

Name		Name	Last commit message	Last commit date
Latest commit History 44 Commits
ML-code		ML-code
Muon		Muon
levanter		levanter
levanter_cats256		levanter_cats256
levanter_og		levanter_og
old_slop		old_slop
psgd_torch		psgd_torch
.gitignore		.gitignore
BatchNormIdentityCReLU.ipynb		BatchNormIdentityCReLU.ipynb
BatchNormResCReLU.ipynb		BatchNormResCReLU.ipynb
CReLU_Expressiveness.ipynb		CReLU_Expressiveness.ipynb
CReLU_Expressiveness_2.ipynb		CReLU_Expressiveness_2.ipynb
CReLU_Memory.ipynb		CReLU_Memory.ipynb
CReLU_Memory_Optim.ipynb		CReLU_Memory_Optim.ipynb
DenseBatchNormCReL4.ipynb		DenseBatchNormCReL4.ipynb
DenseBatchNormCReLU.ipynb		DenseBatchNormCReLU.ipynb
DenseBatchNormCReLU2.ipynb		DenseBatchNormCReLU2.ipynb
DenseBatchNormCReLU3.ipynb		DenseBatchNormCReLU3.ipynb
DenseBatchNormCReLU5.ipynb		DenseBatchNormCReLU5.ipynb
DenseBatchNormCReLU6.ipynb		DenseBatchNormCReLU6.ipynb
DenseCReLU.ipynb		DenseCReLU.ipynb
DenseZeroCReLU.ipynb		DenseZeroCReLU.ipynb
DepthWidthCReLU.ipynb		DepthWidthCReLU.ipynb
FullLayerBatchNorm.ipynb		FullLayerBatchNorm.ipynb
GradientBoostBatchNormCReLU.ipynb		GradientBoostBatchNormCReLU.ipynb
LayerNormIdentity.ipynb		LayerNormIdentity.ipynb
MaxTrainability.ipynb		MaxTrainability.ipynb
MuonDynamicalIsometryCReLU.ipynb		MuonDynamicalIsometryCReLU.ipynb
PairwiseLowOrder.ipynb		PairwiseLowOrder.ipynb
PowerScheduler.ipynb		PowerScheduler.ipynb
README.md		README.md
ReZeroInitCReLU.ipynb		ReZeroInitCReLU.ipynb
TODO.md		TODO.md
TestPReLU.ipynb		TestPReLU.ipynb
TestPReLU2.ipynb		TestPReLU2.ipynb
report.xls		report.xls

Provide feedback