v0.35.0

NOTE: This release is not yet an official ASF release.

Upgrade to v0.35

Public API

• OpenDAL removes rarely used Operator::from_env and Operator::from_iter APIs
  • Users can use Operator::via_map instead.

Raw API

• OpenDAL adds append support with could break existing layers. Please make sure append requests have been forward correctly.
• After the merging of scan and list, OpenDAL removes the scan from raw API. Please use list_without_delimiter instead.

---

v0.35.0 - 2023-05-23

Added

• feat(services/onedrive): Implement list, create_dir, stat and upload
  ing large files (#2231)
• feat(bindings/C): Initially support stat in C binding (#2249)
• feat(bindings/python): Enable abi3 to avoid building on different python
  version (#2255)
• feat(bindings/C): support BDD tests using GTest (#2254)
• feat(services/sftp): setup integration tests (#2192)
• feat(core): Add trait and public API for append (#2260)
• feat(services/sftp): support copy and rename for sftp (#2263)
• feat(services/sftp): support copy and read_seek (#2267)
• feat: Add COS service support (#2269)
• feat(services/cos): Add support for loading from env (#2271)
• feat(core): add presign support for obs (#2253)
• feat(services/sftp): setup integration tests (#2192)
• feat(core): add presign support for obs (#2253)
• feat(core): public API of append (#2284)
• test(core): test for append (#2286)
• feat(services/oss): add append support (#2279)
• feat(bindings/java): implement async ops to pass AsyncStepsTest (#2291)

Changed

• services/gdrive: port code to GdriveCore & add path_2_id cache (#2203)
• refactor: Minimize futures dependencies (#2248)
• refactor: Add Operator::via_map to support init without generic type parameters (#2280)
• refactor(binding/java): build, async and docs (#2276)

Fixed

• fix: Fix bugs that failed wasabi's integration tests (#2273)

Removed

• feat(core): remove scan from raw API (#2262)

Docs

• chore(s3): update builder region doc (#2247)
• docs: Add services in readme (#2251)
• docs: Unify capabilities list for kv services (#2257)
• docs(nodejs): fix some example code errors (#2277)
• docs(bindings/C): C binding contributing documentation (#2266)
• docs: Add new docs that available for all languages (#2285)
• docs: Remove unlicensed svg (#2289)
• fix(website): double active route (#2290)

CI

• ci: Enable test for cos (#2270)
• ci: Add integration tests for supabase (#2272)
• ci: replace set-output for docs (#2275)
• ci: Fix unit tests (#2282)
• ci: Cleanup NOTICE file (#2281)
• ci: Fix release not contains incubating (#2292)

Chore

• chore(core): remove unnecessary path prefix (#2265)

New Contributors

• @saiintbrisson made their first contribution in #2247
• @lqhuang made their first contribution in #2257
• @morristai made their first contribution in #2265
• @C-Dao made their first contribution in #2277

Full Changelog: v0.34.0...v0.35.0

v0.34.0

NOTE: This release is not yet an official ASF release. We are waiting for the vote.

v0.34.0 - 2023-05-09

Added

• feat(writer): configurable buffer size of unsized write (#2143)
• feat(oay): Add basic s3 list_objects_v2 with start_after support (#2219)
• feat: Add typed kv adapter and migrate moka to it (#2222)
• feat: migrate service dashmap (#2225)
• feat(services/memory): migrate service memory (#2229)
• feat: Add assert for public types to ensure Send + Sync (#2237)
• feat(services/gcs): Add abort support for writer (#2242)

Changed

• refactor: Replace futures::ready with std::task::ready (#2221)
• refactor: Use list without delimiter to replace scan (#2243)

Fixed

• fix(services/gcs): checked_rem_euclid could return Some(0) (#2220)
• fix(tests): Etag must be wrapped by " (#2226)
• fix(services/s3): Return error if credential load fail instead skip (#2233)
• fix(services/s3): Return error if region not set for AWS S3 (#2234)
• fix(services/gcs): rsa 0.9 breaks gcs presign (#2236)

Chore

• chore: change log subscriber from env_logger to tracing-subscriber (#2238)
• chore: Fix build of wasabi (#2241)

New Contributors

• @tottoto made their first contribution in #2221

Full Changelog: v0.33.3...v0.34.0

v0.33.3

NOTE: This release is not yet an official ASF release. We are waiting for the vote.

v0.33.3 - 2023-05-06

Added

• feat(services/onedrive): Add read and write support for OneDrive (#2129)
• test(core): test for read_with_override_cache_control (#2155)
• feat(http_util): Implement multipart/form-data support (#2157)
• feat(http_util): Implement multipart/mixed support (#2161)
• RFC-2133: Introduce Append API (#2133)
• feat(services/sftp): Add read/write/stat support for sftp (#2186)
• feat(services/gdrive): Add read & write & delete support for GoogleDrive (#2184)
• feat(services/vercel): Add vercel remote cache support (#2193)
• feat(tests): Enable supabase integration tests (#2190)
• feat(core): merge scan and list (#2214)

Changed

• refactor(java): refactor java code for java binding (#2145)
• refactor(layers/logging): parsing level str (#2160)
• refactor: Move not initiated logic to utils instead (#2196)
• refactor(services/memcached): Rewrite memecached connection entirely (#2204)

Fixed

• fix(service/s3): set retryable on batch (#2171)
• fix(services/supabase): Supabase ci fix (#2200)

Docs

• docs(website): try to add opendal logo (#2159)
• doc: update vision to be more clear (#2164)
• docs: Refactor Contributing and add Developing (#2169)
• docs: Merge DEVELOPING into CONTRIBUTING (#2172)
• docs: fix some grammar errors in the doc of Operator (#2173)
• docs(nodejs): Add CONTRIBUTING docs (#2174)
• docs: Add CONTRIBUTING for python (#2188)

CI

• ci: Use microsoft rust devcontainer instead (#2165)
• ci(devcontainer): Install development deps (#2167)
• chore: set workspace default members (#2168)
• ci: Setup vercel artifacts integration tests (#2197)
• ci: Remove not used odev tools (#2202)
• ci: Add tools to generate NOTICE and all deps licenses (#2205)
• ci: use Temurin JDK 11 to build the bindings-java (#2213)

Chore

• chore(deps): bump clap from 4.1.11 to 4.2.5 (#2183)
• chore(deps): bump futures from 0.3.27 to 0.3.28 (#2181)
• chore(deps): bump assert_cmd from 2.0.10 to 2.0.11 (#2180)
• chore: Refactor behavior test (#2189)
• chore: update readme for more information that is more friendly to newcomers (#2217)

New Contributors

• @SteveLauC made their first contribution in #2173
• @Retrospection made their first contribution in #2193
• @nodece made their first contribution in #2213

Full Changelog: v0.33.2...v0.33.3

v0.33.2

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.33.2 - 2023-04-27

Added

• feat(core): add test for stat_with_if_none_match (#2122)
• feat(services/gcs): Add start-after support for list (#2107)
• feat(services/azblob): Add supporting presign (#2120)
• feat(services/gcs): Add supporting presign support (#2126)
• feat(java): connect rust async/await with java future (#2112)
• docs: add hdfs classpath related troubleshoot (#2130)
• fix(clippy): suppress dead_code check (#2135)
• feat(core): Add cache-control to Metadata (#2136)
• fix(services/gcs): Remove HOST header to avoid gcs RESET connection (#2139)
• test(core): test for write_with_cache_control (#2131)
• test(core): test for write_with_content_type (#2140)
• test(core): test for read_with_if_none_match (#2141)
• feat(services/supabase): Add read/write/stat support for supabase (#2119)

Docs

• docs: add hdfs classpath related troubleshoot (#2130)

CI

• ci: Mark job as skipped if owner is not apache (#2128)
• ci: Enable native-tls to test gcs presign for workaround (#2138)

New Contributors

• @ShadowySpirits made their first contribution in #2112

Full Changelog: v0.33.1...v0.33.2

v0.33.1

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.33.1 - 2023-04-25

Added

• feat: Add behavior test for read_with_if_match & stat_with_if_match (#2088)
• feat(tests): Add fuzz test for writer without content length (#2100)
• feat: add if_none_match support for obs (#2103)
• feat(services/oss): Add server side encryption support for oss (#2092)
• feat(core): update errorKind PreconditionFailed to ConditionNotMatch (#2104)
• feat(services/s3): Add start-after support for list (#2096)
• feat: gcs support cache control (#2116)

Fixed

• fix(services/gcs): set content length=0 for gcs initiate_resumable_upload (#2110)
• fix(bindings/nodejs): Fix index.d.ts not updated (#2117)

Docs

• chore: improve LoggingLayer docs and pub use log::Level (#2089)
• docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)

CI

• chore(bindings/nodejs): update package.json repository info (#2078)
• ci: Bring hdfs test back (#2114)

New Contributors

• @junaire made their first contribution in #2103

Full Changelog: v0.33.0...v0.33.1

v0.33.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.33

Public API

OpenDAL 0.33 has redesigned the Writer API, replacing all instances of writer.append() with writer.write(). For more information, please refer to Writer.

Raw API

In addition to the redesign of the Writer API, we have removed append from oio::Write. Therefore, users who implement services and layers should also remove it.

After v0.33 landing, services should handle OpWrite::content_length correctly by following these guidelines:

• If the writer does not support uploading unsized data, return a response of NotSupported if content length is None.
• Otherwise, continue writing data until either close or abort has been called.

Furthermore, OpenDAL 0.33 introduces a new concept called Capability which replaces AccessorCapability. Services must adapt to this change.

---

v0.33.0 - 2023-04-23

Added

• feat: Add OpenTelemetry Trace Layer (#2001)
• feat: add if_none_match support for azblob (#2035)
• feat: add if_none_match/if_match for gcs (#2039)
• feat: Add size check for sized writer (#2038)
• feat(services/azblob): Add if-match support (#2037)
• feat(core): add copy&rename to error_context layer (#2040)
• feat: add if-match support for OSS (#2034)
• feat: Bootstrap new (old) project oay (#2041)
• feat(services/OSS): Add override_content_disposition support (#2043)
• feat: add IF_MATCH for http (#2044)
• feat: add IF_MATCH for http HEAD request (#2047)
• feat: add cache control header for azblob and obs (#2049)
• feat: Add capability for operation's variant and args (#2057)
• feat(azblob): Add override_content_disposition support (#2065)
• feat(core): test for read_with_override_content_composition (#2067)
• feat(core): Add start-after support for list (#2071)

Changed

• refactor: Polish Writer API by merging append and write together (#2036)
• refactor(raw/http_util): Add url in error context (#2066)
• refactor: Allow reusing the same operator to speed up tests (#2068)

Fixed

• fix(bindings/ruby): use rb_sys_env to help find ruby for building (#2051)
• fix: MadsimLayer should be able to built without cfg (#2059)
• fix(services/s3): Ignore prefix if it's empty (#2064)

Docs

• docs(bindings/python): ipynb examples for users (#2061)

CI

• ci(bindings/nodejs): publish support --provenance (#2046)
• ci: upgrade typos to 1.14.8 (#2055)
• chore(bindings/C): ignore the formatting of auto-generated opendal.h (#2056)

New Contributors

• @leenstx made their first contribution in #2033
• @Essoz made their first contribution in #2035
• @imp2002 made their first contribution in #2037
• @Renkai made their first contribution in #2034
• @Weijun-H made their first contribution in #2044
• @everpcpc made their first contribution in #2051
• @Justin-Xiang made their first contribution in #2061
• @lexcao made their first contribution in #2071

Full Changelog: v0.32.0...v0.33.0

v0.32.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.32

OpenDAL 0.32 doesn't have much breaking changes.

We changed Accessor::create into Accessor::create_dir. Only users who implement Layer need to change.

v0.32.0 - 2023-04-18

Added

• feat: Add wasabi service implementation (#2004)
• feat: improve the readability of oli command line error output (#2016)
• feat: add If-Match Support for OpRead, OpWrite, OpStat (#2017)
• feat: add behavioral test for Write::abort (#2018)
• feat: add if-match support for obs (#2023)
• feat: Add missing functions for trace layers (#2025)
• feat(layer): add madsim layer (#2006)
• feat(gcs): add support for gcs append (#1801)

Changed

• refactor: Rename Create to CreateDir for its behavior changed (#2019)

Fixed

• fix: Cargo lock not updated (#2027)
• fix(services/s3): Ignore empty query to make it more compatible (#2028)
• fix(services/oss): Fix env not loaded for oss signer (#2029)

Docs

• docs: fix some typos (#2022)
• docs: add dev dependency section (#2021)

New Contributors

• @czybjtu made their first contribution in #2016
• @Yansongsongsong made their first contribution in #2022
• @haohuaijin made their first contribution in #2023
• @SkyFan2002 made their first contribution in #2006
• @cuichenli made their first contribution in #1801

Full Changelog: v0.31.1...v0.32.0

v0.31.1

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.31.1 - 2023-04-17

Added

• feat(services/azdfs): support rename (#1929)
• test: Increate copy/move nested path test depth (#1932)
• feat(layers): add a basic minitrace layer (#1931)
• feat: add Writer::abort method (#1937)
• feat(services/gcs): Allow setting PredefinedAcl (#1989)
• feat(services/oss): add oss cache-control header support (#1986)
• feat: Add PreconditionFailed Error so users can handle them (#1993)
• feat: add http if_none_match support (#1995)
• feat: add oss if-none-match support (#1997)
• feat(services/gcs): Allow setting default storage_class (#1996)
• feat(binding/C): add clang-format for c binding (#2003)

Changed

• refactor: Polish the behavior of scan (#1926)
• refactor: Polish the implementation of webhdfs (#1935)

Fixed

• fix: sled should not be enabled by default (#1923)
• fix: kv adapter's writer implementation fixed to honour empty writes (#1934)
• fix(services/azblob): fix copy missing content-length (#2000)

Docs

• docs: Adding docs link to python binding (#1921)
• docs(bindings/python): fix wrong doc link (#1928)
• docs: Add contributing for OpenDAL (#1984)
• docs: Add explanation in contributing (#1985)
• docs: Feel relax in community and don't hurry (#1987)
• docs: update contributing (#1998)
• docs(services/memory): Fix wrong explanation (#2002)
• docs: Add OpenDAL VISION (#2007)
• docs: update VISION and behavior tests doc (#2009)

CI

• ci(bindings/nodejs): Access should be set to public before publish (#1919)
• ci: Re-enable webhdfs test (#1925)
• chore: add .editorconfig (#1988)
• ci: Fix format after adding editorconfig (#1990)

New Contributors

• @sa- made their first contribution in #1921
• @ahonn made their first contribution in #1988
• @silver-ymz made their first contribution in #1989
• @Nickqiaoo made their first contribution in #1995
• @thorseraq made their first contribution in #1997

Full Changelog: v0.31.0...v0.31.1

v0.31.0

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

Upgrade to v0.31

In version v0.31 of OpenDAL, we made some internal refactoring to improve its compatibility with the ecosystem.

MSRV Bump

We increased the MSRV to 1.64 from v0.31 onwards. Although it is still possible to build OpenDAL under older rustc versions, we cannot guarantee that any issues related to them will be fixed.

Accept std::time::Duration instead

Previously, OpenDAL accepted time::Duration as input for presign_xxx. However, since v0.31, we have changed this to accept std::time::Duration so that users do not need to depend on time. Internally, we migrated from time to chrono for better integration with other parts of the ecosystem.

disable_ec2_metadata for services s3

We have added a new configuration option called disable_ec2_metadata for the S3 service in response to a mistake where it was mixed up with another option called disable_config_load. Users who want to disable loading credentials from EC2 metadata should set this option instead.

Services Feature Flag

Starting from v0.31, all services in OpenDAL are split into different feature flags. To enable only S3 support, use the following TOML configuration:

opendal = {
    version = "0.31",
    default-features = false,
    features = ["services-s3"]
}

---

v0.31.0 - 2023-04-12

Added

• feat(bindings/java): add cucumber test (#1809)
• feat(bindings/java): setup Java CI (#1823)
• feat: add if_none_match support (#1832)
• feat: Retry when some of batch operations are failing (#1840)
• feat: add writer support for aliyun oss (#1842)
• feat(core): Add Copy Support (#1841)
• feat(bindings/c): fix c bindings makefile (#1849)
• feat(core): add behavior tests for copy & blocking_copy (#1852)
• feat(s3): allow users to specify storage_class (#1854)
• feat(s3): Support copy (#1856)
• Add check for s3 bucket name (#1857)
• feat(core): Support rename (#1862)
• feat(bindings/nodejs): add copy and rename (#1866)
• feat(azblob): Support copy (#1868)
• feat(gcs): copy support for GCS (#1869)
• feat(bindings/c): framework of add basic io and init logics (#1861)
• feat(webdav): support copy (#1870)
• feat(services/oss): Add Copy Support (#1874)
• feat(services/obs): Add Copy Support (#1876)
• feat(services/webdav): Support Rename (#1878)
• binding/c: parse opendal to use typed BlockingOperator (#1881)
• binding/c: clean up comments and type assertion for BlockingOperator (#1883)
• binding(python): Support python binding benchmark for opendal (#1882)
• feat(bindings/c): add support for free heap-allocated operator (#1890)
• feat(binding/c): add is_exist to operator (#1892)
• feat(bindings/java): add Stat support (#1894)
• feat(services/gcs): Add customed token loader support (#1908)
• feat(services/oss): remove unused builder prop allow_anonymous (#1913)
• feat: Add feature flag for all services (#1915)

Changed

• refactor(core): Simplify the usage of BatchOperation and BatchResults (#1843)
• refactor: Use reqwest blocking client instead of ureq (#1853)
• refactor: Bump MSRV to 1.64 (#1864)
• refactor: Remove not used blocking http client (#1895)
• refactor: Change presign to async for future refactor (#1900)
• refactor(services/gcs): Migrate to async reqsign (#1906)
• refactor(services/azdfs): Migrate to async reqsign (#1903)
• refactor(services/azblob): Adopt new reqsign (#1902)
• refactor(services/s3): Migrate to async reqsign (#1909)
• refactor(services/oss): Migrate to async reqsign (#1911)
• refactor: Use chrono instead of time to work well with ecosystem (#1912)
• refactor(service/obs): Migrate obs to async reqsign (#1914)

Fixed

• fix: podling website check (#1838)
• fix(website): copyright update (#1839)
• fix(core): Add checks before doing copy (#1845)
• fix(core): S3 Copy should set SSE headers (#1860)
• fix: Fix presign related unit tests (#1910)

Docs

• docs(bindings/nodejs): fix build failed (#1819)
• docs: fix several typos in the documentation (#1846)
• doc(bindings/nodejs): update presign example in doc (#1901)

CI

• ci: Fix build for nodejs binding on macos (#1813)
• binding/c: build: add phony to makefile, and some improve (#1850)
• ci: upgrade hawkeye action (#1834)

Chore

• chore(bindings/nodejs): add deno benchmark (#1814)
• chore: Add CODEOWNERS (#1817)
• chore(deps): bump opentelemetry-jaeger from 0.16.0 to 0.18.0 (#1829)
• chore(deps): bump opentelemetry from 0.17.0 to 0.19.0 (#1830)
• chore(deps): bump tokio from 1.26.0 to 1.27.0 (#1828)
• chore(deps): bump napi-derive from 2.12.1 to 2.12.2 (#1827)
• chore(deps): bump async-trait from 0.1.67 to 0.1.68 (#1826)
• chore: Cleanup code for oss writer (#1847)
• chore: Make clippy happy (#1865)
• binding(python): Format python code in binding (#1885)

New Contributors

• @gitccl made their first contribution in #1832
• @v0y4g3r made their first contribution in #1842
• @Ji-Xinyou made their first contribution in #1846
• @darknight made their first contribution in #1857
• @Zheaoli made their first contribution in #1882

Full Changelog: v0.30.5...v0.31.0

v0.30.5

NOTE: This release is not yet an official ASF release. We are still in the process of learning how to create a formal one.

v0.30.5 - 2023-03-31

Added

• feat(oli): implement oli rm (#1774)
• feat(bindings/nodejs): Support presign (#1772)
• feat(oli): implement oli stat (#1778)
• feat(bindings/object_store): Add support for list and list_with_delimiter (#1784)
• feat(oli): implement oli cp -r (#1787)
• feat(bindings/nodejs): Make PresignedRequest serializable (#1797)
• feat(binding/c): add build.rs and cbindgen as dep to gen header (#1793)
• feat(bindings/nodejs): Add more APIs and examples (#1799)
• feat: reader_with and writer_with (#1803)
• feat: add override_cache_control (#1804)
• feat: add cache_control to OpWrite (#1806)

Changed

• refactor(oli): switch to Operator::scan and Operator::remove_all (#1779)
• refactor(bindings/nodejs): Polish benchmark to make it more readable (#1810)

Fixed

• fix(oli): set the root of fs service to '/' (#1773)
• fix: align WebDAV stat with RFC specification (#1783)
• fix(bindings/nodejs): fix read benchmark (#1805)

CI

• ci: Split clippy and docs check (#1785)
• ci(bindings/nodejs): Support aarch64-apple-darwin (#1780)
• ci(bindings/nodejs): publish with LICENSE & NOTICE (#1790)
• ci(services/redis): Add dragonfly test (#1808)

Chore

• chore(bindings/python): update maturin to 0.14.16 (#1777)
• chore(bin/oli): Set oli version from package version (#1786)
• chore(oli): set cli version in a central place (#1789)
• chore: don't pin time version (#1788)
• chore(bindings/nodejs): init benchmark (#1798)
• chore(bindings/nodejs): Fix generated headers (#1802)

New Contributors

• @bxb100 made their first contribution in #1783
• @nasen23 made their first contribution in #1784
• @promer94 made their first contribution in #1798

Full Changelog: v0.30.4...v0.30.5