Skip to content

v0.3.0

Latest
Latest
Compare
Choose a tag to compare
@Xuanwo Xuanwo released this 19 Aug 10:20
· 75 commits to main since this release
v0.3.0
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Learn about vigilant mode.
79711ac
This commit was created on github.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

What's Changed

  • Smooth out release steps by @Fokko in #197
  • refactor: remove support of manifest list format as a list of file path by @Dysprosium0626 in #201
  • refactor: remove unwraps by @odysa in #196
  • Fix: add required rust version in cargo.toml by @dp-0 in #193
  • Fix the REST spec version by @Fokko in #198
  • feat: Add Sync + Send to Catalog trait by @ZhengLin-Li in #202
  • feat: Make thrift transport configurable by @DeaconDesperado in #194
  • Add UnboundSortOrder by @fqaiser94 in #115
  • ci: Add workflow for publish by @Xuanwo in #218
  • Add workflow for cargo audit by @sdd in #217
  • docs: Add basic README for all crates by @Xuanwo in #215
  • Follow naming convention from Iceberg's Java and Python implementations by @s-akhtar-baig in #204
  • doc: Add download page by @liurenjie1024 in #219
  • chore(deps): Update derive_builder requirement from 0.13.0 to 0.20.0 by @dependabot in #203
  • test: add FileIO s3 test by @odysa in #220
  • ci: Ignore RUSTSEC-2023-0071 for no actions to take by @Xuanwo in #222
  • feat: Add expression builder and display. by @liurenjie1024 in #169
  • chord: Add IssueNavigationLink for RustRover by @stream2000 in #230
  • minor: Fix double API doc by @viirya in #226
  • feat: add UnboundPredicate::negate() by @sdd in #228
  • fix: Remove deprecated methods to pass ci by @liurenjie1024 in #234
  • Implement basic Parquet data file reading capability by @sdd in #207
  • chore: doc-test as a target by @liurenjie1024 in #235
  • feat: add parquet writer by @ZENOTME in #176
  • Add hive metastore catalog support (part 1/2) by @marvinlanhenke in #237
  • chore: Enable projects. by @liurenjie1024 in #247
  • refactor: Make plan_files as asynchronous stream by @viirya in #243
  • feat: Implement binding expression by @liurenjie1024 in #231
  • Implement Display instead of ToString by @lewiszlw in #256
  • add rewrite_not by @marvinlanhenke in #263
  • feat: init TableMetadataBuilder by @ZENOTME in #262
  • Rename stat_table to table_exists in Catalog trait by @lewiszlw in #257
  • feat (static table): implement a read-only table struct loaded from metadata by @a-agmon in #259
  • feat: implement OAuth for catalog rest client by @TennyZhuang in #254
  • docs: annotate precision and length to primitive types by @waynexia in #270
  • build: Restore CI by making parquet and arrow version consistent by @viirya in #280
  • Metadata Serde + default partition_specs and sort_orders by @marvinlanhenke in #272
  • feat: make optional oauth param configurable by @himadripal in #278
  • fix: enable public access to ManifestEntry properties by @a-agmon in #284
  • feat: Implement the conversion from Arrow Schema to Iceberg Schema by @viirya in #258
  • Rename function name to add_manifests by @viirya in #293
  • Modify Bind calls so that they don't consume self and instead return a new struct, leaving the original unmoved by @sdd in #290
  • Add hive metastore catalog support (part 2/2) by @marvinlanhenke in #285
  • feat: implement prune column for schema by @Dysprosium0626 in #261
  • chore(deps): Update reqwest requirement from ^0.11 to ^0.12 by @dependabot in #296
  • Glue Catalog: Basic Setup + Test Infra (1/3) by @marvinlanhenke in #294
  • feat: rest client respect prefix prop by @TennyZhuang in #297
  • fix: HMS Catalog missing properties fn create_namespace by @marvinlanhenke in #303
  • fix: renaming FileScanTask.data_file to data_manifest_entry by @a-agmon in #300
  • feat: Make OAuth token server configurable by @whynick1 in #305
  • feat: Glue Catalog - namespace operations (2/3) by @marvinlanhenke in #304
  • feat: add transform_literal by @ZENOTME in #287
  • feat: Complete predicate builders for all operators. by @QuakeWang in #276
  • feat: Support customized header in Rest catalog client by @whynick1 in #306
  • fix: chrono dep by @odysa in #274
  • feat: Read Parquet data file with projection by @viirya in #245
  • Fix day timestamp micro by @marvinlanhenke in #312
  • feat: support uri redirect in rest client by @TennyZhuang in #310
  • refine: seperate parquet reader and arrow convert by @ZENOTME in #313
  • chore: upgrade to rust-version 1.77.1 by @marvinlanhenke in #316
  • Support identifier warehouses by @Fokko in #308
  • feat: Project transform by @marvinlanhenke in #309
  • Add Struct Accessors to BoundReferences by @sdd in #317
  • Use str args rather than String in transform to avoid needing to clone strings by @sdd in #325
  • chore(deps): Update pilota requirement from 0.10.0 to 0.11.0 by @dependabot in #327
  • chore(deps): Bump peaceiris/actions-mdbook from 1 to 2 by @dependabot in #332
  • chore(deps): Bump peaceiris/actions-gh-pages from 3.9.3 to 4.0.0 by @dependabot in #333
  • chore(deps): Bump apache/skywalking-eyes from 0.5.0 to 0.6.0 by @dependabot in #328
  • Add BoundPredicateVisitor (alternate version) by @sdd in #334
  • add InclusiveProjection Visitor by @sdd in #335
  • feat: Implement the conversion from Iceberg Schema to Arrow Schema by @ZENOTME in #277
  • Simplify expression when doing {and,or} operations by @Fokko in #339
  • feat: Glue Catalog - table operations (3/3) by @marvinlanhenke in #314
  • chore: update roadmap by @marvinlanhenke in #336
  • Add ManifestEvaluator, used to filter manifests in table scans by @sdd in #322
  • feat: init iceberg writer by @ZENOTME in #275
  • Implement manifest filtering in TableScan by @sdd in #323
  • Refactor: Extract partition_filters from ManifestEvaluator by @marvinlanhenke in #360
  • Basic Integration with Datafusion by @marvinlanhenke in #324
  • refactor: cache partition_schema in fn plan_files() by @marvinlanhenke in #362
  • fix (manifest-list): added serde aliases to support both forms conventions by @a-agmon in #365
  • feat: Extract FileRead and FileWrite trait by @Xuanwo in #364
  • feat: Convert predicate to arrow filter and push down to parquet reader by @viirya in #295
  • chore(deps): Update datafusion requirement from 37.0.0 to 38.0.0 by @dependabot in #369
  • chore(deps): Update itertools requirement from 0.12 to 0.13 by @dependabot in #376
  • Add InclusiveMetricsEvaluator by @sdd in #347
  • Rename V2 spec names by @gupteaj in #380
  • feat: make file scan task serializable by @ZENOTME in #377
  • Feature: Schema into_builder method by @c-thiel in #381
  • replaced i32 in TableUpdate::SetDefaultSortOrder to i64 by @rwwwx in #387
  • fix: make PrimitiveLiteral and Literal not be Ord by @ZENOTME in #386
  • docs(writer/docker): fix small typos and wording by @jdockerty in #389
  • feat: StructAccessor.get returns Result<Option<Datum>> instead of Result<Datum> by @sdd in #390
  • feat: add ExpressionEvaluator by @marvinlanhenke in #363
  • Derive Clone for TableUpdate by @c-thiel in #402
  • Add accessor for Schema identifier_field_ids by @c-thiel in #388
  • deps: Bump arrow related crates to 52 by @Dysprosium0626 in #403
  • SnapshotRetention::Tag max_ref_age_ms should be optional by @c-thiel in #391
  • feat: Add storage features for iceberg by @Xuanwo in #400
  • Implement BoundPredicateVisitor trait for ManifestFilterVisitor by @s-akhtar-baig in #367
  • Add missing arrow predicate pushdown implementations for StartsWith, NotStartsWith, In, and NotIn by @sdd in #404
  • feat: make BoundPredicate,Datum serializable by @ZENOTME in #406
  • refactor: Upgrade hive_metastore to 0.1 by @Xuanwo in #409
  • fix: Remove duplicate filter by @liurenjie1024 in #414
  • Enhancement: refine the reader interface by @ZENOTME in #401
  • refactor(catalog/rest): Split http client logic to seperate mod by @Xuanwo in #423
  • Remove #[allow(dead_code)] from the codebase by @vivek378521 in #421
  • ci: use official typos github action by @shoothzj in #426
  • feat: support lower_bound&&upper_bound for parquet writer by @ZENOTME in #383
  • refactor: Implement ArrowAsyncFileWriter directly to remove tokio by @Xuanwo in #427
  • chore: Don't enable reqwest default features by @Xuanwo in #432
  • refactor(catalogs/rest): Split user config and runtime config by @Xuanwo in #431
  • feat: runtime module by @odysa in #233
  • fix: Fix namespace identifier in url by @liurenjie1024 in #435
  • refactor(io): Split io into smaller mods by @Xuanwo in #438
  • chore: Use once_cell to replace lazy_static by @Xuanwo in #443
  • fix: Fix build while no-default-features enabled by @Xuanwo in #442
  • chore(deps): Bump crate-ci/typos from 1.22.9 to 1.23.1 by @dependabot in #447
  • docs: Refactor the README to be more user-oriented by @Xuanwo in #444
  • feat: Add cargo machete by @vaibhawvipul in #448
  • chore: Use nightly toolchain for check by @liurenjie1024 in #445
  • reuse docker container to save compute resources by @thexiay in #428
  • feat: Add macos runner for ci by @QuakeWang in #441
  • chore: remove compose obsolete version (#452) by @yinheli in #454
  • Refactor file_io_s3_test.rs by @fqaiser94 in #455
  • chore(deps): Bump crate-ci/typos from 1.23.1 to 1.23.2 by @dependabot in #457
  • refine: move binary serialize in literal to datum by @ZENOTME in #456
  • fix: Hms test on macos should use correct arch by @liurenjie1024 in #461
  • Fix ManifestFile length calculation by @nooberfsh in #466
  • chore(deps): Update typed-builder requirement from ^0.18 to ^0.19 by @dependabot in #473
  • fix: use avro fixed to represent decimal by @xxchan in #472
  • feat(catalog!): Deprecate rest.authorization-url in favor of oauth2-server-uri by @ndrluis in #480
  • Alter Transform::Day to map partition types to Date rather than Int for consistency with reference implementation by @sdd in #479
  • feat(iceberg): Add memory file IO support by @Xuanwo in #481
  • Add memory catalog implementation by @fqaiser94 in #475
  • chore: Enable new rust code format settings by @Xuanwo in #483
  • docs: Generate rust API docs by @Xuanwo in #486
  • chore: Fix format of recent PRs by @Xuanwo in #487
  • Rename folder to memory by @fqaiser94 in #490
  • chore(deps): Bump crate-ci/typos from 1.23.2 to 1.23.5 by @dependabot in #493
  • View Spec implementation by @c-thiel in #331
  • fix: Return error on reader task by @ndrluis in #498
  • chore: Bump OpenDAL to 0.48 by @Xuanwo in #500
  • feat: add check compatible func for primitive type by @ZENOTME in #492
  • refactor(iceberg): Remove an extra config parse logic by @Xuanwo in #499
  • feat: permit Datum Date<->Int type conversion by @sdd in #496
  • Add additional S3 FileIO Attributes by @c-thiel in #505
  • docs: Add links to dev docs by @Xuanwo in #508
  • chore: Remove typo in README by @Xuanwo in #509
  • feat: podman support by @alexyin1 in #489
  • feat(table): Add debug and clone trait to static table struct by @ndrluis in #510
  • Use namespace location or warehouse location if table location is missing by @fqaiser94 in #511
  • chore(deps): Bump crate-ci/typos from 1.23.5 to 1.23.6 by @dependabot in #521
  • Concurrent table scans by @sdd in #373
  • refactor: replace num_cpus with thread::available_parallelism by @SteveLauC in #526
  • Fix: MappedLocalTime should not be exposed by @c-thiel in #529
  • feat: Establish subproject pyiceberg_core by @Xuanwo in #518
  • fix: complete miss attribute for map && list in avro schema by @ZENOTME in #411
  • arrow/schema.rs: refactor tests by @AndreMouche in #531
  • feat: initialise SQL Catalog by @callum-ryan in #524
  • chore(deps): Bump actions/setup-python from 4 to 5 by @dependabot in #536
  • feat(storage): support aws session token by @twuebi in #530
  • Simplify PrimitiveLiteral by @ZENOTME in #502
  • chore: bump opendal to 0.49 by @jdockerty in #540
  • feat: support timestamp columns in row filters by @sdd in #533
  • fix: don't silently drop errors encountered in table scan file planning by @sdd in #535
  • chore(deps): Update sqlx requirement from 0.7.4 to 0.8.0 by @dependabot in #537
  • Fix main branch building break by @liurenjie1024 in #541
  • feat: support for gcs storage by @jdockerty in #520
  • feat: Allow FileIO to reuse http client by @Xuanwo in #544
  • docs: Add an example to scan an iceberg table by @Xuanwo in #545
  • Concurrent data file fetching and parallel RecordBatch processing by @sdd in #515
  • doc: Add statement for contributors to avoid force push as much as possible by @liurenjie1024 in #546
  • chore: Bump datafusion to 41 by @Xuanwo in #548
  • feat: Partition Binding and safe PartitionSpecBuilder by @c-thiel in #491
  • Bump to version 0.3.0 by @Xuanwo in #549

New Contributors

Full Changelog: 0.2.0...v0.3.0

Contributors

viirya, yinheli, and 38 other contributors
Assets 2
Loading