test: add arrow datasets and arrow unit tests#403

Merged

ashokponkumar merged 6 commits intofoundation-model-stack:mainfrom willmj:arrow-format

Dec 7, 2024

+50

Commits on Dec 4, 2024

test: add arrow datasets and arrow unit tests
willmj
committed

Commits on Dec 5, 2024

test: pretokenized arrow dataset
willmj
committed

Commits on Dec 6, 2024

fmt
willmj
committed

Commits on Dec 7, 2024