Tracking issue: Datastore performance/QOL improvements

[teh-cmc]

All kinds of improvements to the store that would be very welcome although not particularly critical/urgent.

• Garbage collection should be aware of app_id/recording_id semantics #1904
• Rebatch aggressively while dumping the store to a stream of LogMsg #1894
• Optimize text logs #1887
• Make log_time column implicit and potentially introduce ingest_time #1891
• datastore: workload specific GC optimizations #1823
• Awkward datagen module needs to go away #1810
• Implement minimal datatype registry for cross-batch deduplication #1809
• datastore: timeless ordering should be RowId-based #1807
  • arrow2 does _not_ refcount schema metadata #1805
• datastore: GC shouldn't have to return dropped RowIds #1804
• datastore: don't dump autogenerated keys #1793
• datastore: implement shared slices for autogenerated keys #1777
• datastore: rewrite dataframe integration in terms of DataTable's #1759
  • Dataframe extension for Chunk #1692
• Range query misses data when entity is logged with multiple log calls #1218
• Loading of data is extremely slow in web on Chrome #963
• Add splats to polars_util.rs #743
• re_datastore: index bucket compaction #512
• re_datastore: optimize backwards walk in read path #453
• re_datastore: generalized deduplication #446
• re_datastore: implement efficient reshuffling on the read path #442
• re_datastore: properly handle already sorted data during insertion #433
• Efficient DataTable::sort shared with DataStore #1981