Explicit automatic alignment of header

[mrsteyk]

My use case might be overly specific, but when writing/using vectorised code on mmap'd safetensors file header sometimes causes everything to have an odd-numbered pointer which breaks even 16 bit vectorisation. Is there a possibility that python bindings will get an option to save with an explicit alignment? Just padding the header should be enough for most use cases.

[Narsil]

No it's not overly specific, actually there's already a PR for that.

#148

I was waiting for more need for it before merging, but it seems this is picking up in low level frameworks where alignment could really help speed up load times.

[Narsil]

Also I love the project !

If you want pure rust ML framework I recommend https://github.com/coreylowman/dfdx (Still very early on, but there's at least a lot to inspire from IMO).

For instance I implemented https://github.com/Narsil/fast_gpt2 (without dfdx, more like your approach, but still stealing the mkl bindings from dfdx to get the performance ! )

[mrsteyk]

Thank you, @Narsil! Yeah, doing math low level isn't that popular apart from people who know how to code and are on "sub-par" HW by today's standards. Also thanks for the mention of dfdx and your repo. I didn't even consider trying to use any BLAS lib.

[Narsil]

isn't that popular apart from people who know how to code and are on "sub-par" HW by today's standards.

It's still the future in my eyes. The ML fields is somewhat settling and not experimenting as much as it used to, performance is becoming a real concern for anything at scale. And all the python solution for performance are way too clunky to beat compiled code.
This is a very personal view.

[mrsteyk]

Closing because #148 is merged