Weekly Plan: Andrew Lamb 2025-05-05

[alamb]

This is my plan this week for reviews, etc. I am putting it here to make it visible and keep myself organized

• DataFusion Pushdown: use FilterPushdown in Implement Parquet filter pushdown via new filter pushdown APIs #15769 from @adriangb
• DataFusion: aggregate performance: re-review aggregate performance from @Rachelint Intermediate result blocked approach to aggregation memory management #15591
• object_store: fix/merge thread pool PR feat: Add SpawnService and SpawnedReqwestConnector for running requests on a different runtime arrow-rs-object-store#332 from @ion-elgreco
• DataFusion: Min/Max for lists/ nested types: feat(datafusion-functions-aggregate): add support for lists and other nested types in min and max #15857
• Arrow Filter Performance: work on clickbench derived benchmark with @zhuqi-lucas
• Arrow Filter ClickBench benchmark: Add arrow_reader_clickbench benchmark arrow-rs#7470
• ObjectStore release: Release object_store 0.12.1 (non breaking) Around April 30 2025 arrow-rs-object-store#287
• Arrow Release: Release arrow-rs / parquet Minor version 55.1.0 (May 2025) arrow-rs#7393
• Arrow Variant: re-review Review changes to API for variant: Add API for Creating Variant Values arrow-rs#7452 from @PinkCrow007
• Arrow Variant: Create parquet-variant create skeleton PR : Variant: Rust API to Read Variant Values arrow-rs#7423
• ObjectStore: review upstreaming `ObjectStoreRegistry: Add ObjectStoreRegistry arrow-rs-object-store#348
• Arrow Bug with concat'ing dictionaries from @davidhewitt: support merging primitive dictionaries in interleave and concat arrow-rs#7468
• DataFusion: Metadata Handling / extension types from @timsaucer: feat: metadata handling for aggregates and window functions #15911
• Arrow: Help fix duration fix duration conversion error arrow-rs#5626
• DataFusion perf script from @logan-keede : Shell script to collect benchmarks for multiple versions #15144
• DataFusion perf script draft: feat(benchmark): collect benchmarks for last 5 versions in line protocol format #15846
• DataFusion PR about pruning ordering: pipe column orderings into pruning predicate creation #15821
• DataFusion Sorting: test: add fuzz test for doing aggregation with larger than memory groups and sorting with limited memory #15727
• Arrow Dictionary ID next steps from @brancz: Remove dict_id from schema and make it an IPC concern only arrow-rs#7467
• Arrow Variant: Expose tape decoder Add custom decoder in arrow-json arrow-rs#7442

Nice to have (really would be great to have someone help review):

• DataFusion: Aggregate UDFs in FFI: feat: Add Aggregate UDF to FFI crate #14775
• Arrow: Avro cleanup: Avro codec enhancements arrow-rs#6965
• Arrow: Avro Utf8View: Support Utf8View for Avro arrow-rs#7434

[alamb]

Next week:

• Weekly Plan: Andrew Lamb 2025-05-12 #16022