v0.5.0

This release introduces the following new features:

Object::create API

Now, we can create an empty file or directory without a trick!

Read Object::create to know more!

Create empty file

let o = op.object("path/to/file");
let _ = o.create().await?;

Create dir

let o = op.object("path/to/dir/");
let _ = o.create().await?;

Native decompress read support

Now, we can read a compressed file natively!

Enable compress features:

opendal = {version="0.5.0", feautres=["compress"]}

Read with decompress_read() or decompress_reader()!

let o = op.object("path/to/file.gz");
let bs = o.decompress_read().await?;

Azblob support

With the help from @D2Lark, we have official support for azblob now!

Refer to azblob for more information.

// Create azblob backend builder.
let mut builder: Builder = azblob::Backend::build();
// Set the root for azblob, all operations will happen under this root.
//
// NOTE: the root must be absolute path.
builder.root("/path/to/dir");
// Set the container name, this is required.
builder.container("test");
// Set the endpoint, this is required.
//
// For examples:
// - "http://127.0.0.1:10000/devstoreaccount1"
// - "https://accountname.blob.core.windows.net"
builder.endpoint("http://127.0.0.1:10000/devstoreaccount1");
// Set the account_name and account_key.
//
// OpenDAL will try load credential from the env.
// If credential not set and no valid credential in env, OpenDAL will
// send request without signing like anonymous user.
builder.account_name("devstoreaccount1");
builder.account_key("Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==");
// Build the `Accessor`.
let accessor: Arc<dyn Accessor> = builder.finish().await?;

// `Accessor` provides the low level APIs, we will use `Operator` normally.
let op: Operator = Operator::new(accessor);

---

What's Changed

• feat: Improve error message by @Xuanwo in #220
• RFC-0221: Create Dir by @Xuanwo in #221
• feat: Implement RFC-0221 Create Dir by @Xuanwo in #223
• refactor: Move op.objects() to o.list() by @Xuanwo in #224
• feat: Simplify create API by @Xuanwo in #225
• feat: Implement decompress read support by @Xuanwo in #227
• fix: Azblob should pass all behavior tests now by @Xuanwo in #228
• ci: Enable behavior test for azblob by @Xuanwo in #229
• docs: Add docs for azblob's public structs by @Xuanwo in #230
• refactor: Improve behavior_tests so that cargo test works without --all-features by @Xuanwo in #231
• Bump to version 0.5.0 by @Xuanwo in #232

Full Changelog: v0.4.2...v0.5.0

v0.4.2

What's Changed

• feat: Add seekable_reader on Object by @Xuanwo in #215
• fix: Object last_modified should carry timezone by @Xuanwo in #217
• Bump to version 0.4.2 by @Xuanwo in #216

Full Changelog: v0.4.1...v0.4.2

v0.4.1

What's Changed

• docs: Fix typo in CHANGELOG by @Xuanwo in #211
• feat: Export SeekableReader by @Xuanwo in #212
• Bump to version 0.4.1 by @Xuanwo in #213

Full Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

• refactor: Make read return BytesStream instead by @Xuanwo in #192
• feat(services/azblob): Implement list support by @D2Lark in #193
• refactor: Make write return BytesSink instead by @Xuanwo in #194
• feat: Implement io_util like into_sink and into_stream by @Xuanwo in #197
• feat(io_util): Implement observer for sink and stream by @Xuanwo in #198
• RFC-0191: Async Streaming IO by @Xuanwo in #191
• docs: Add docs for all newly added public functions by @Xuanwo in #199
• refactor: New public API design by @Xuanwo in #201
• deps: Bump to reqsign to fix s3 url encode issue by @Xuanwo in #202
• RFC-0203: Remove Credential by @Xuanwo in #203
• refactor: Adopt io::Result instead by @Xuanwo in #204
• refactor: Rollback changes around async streaming io by @Xuanwo in #205
• docs: Add docs for public types by @Xuanwo in #206
• refactor: Refactor behavior tests with macro_rules by @Xuanwo in #207
• Bump to version 0.4.0 by @Xuanwo in #208

Full Changelog: v0.3.0...v0.4.0

---

Refer to Upgrade From v0.3 to v0.4 section for more upgrade details.

From v0.3 to v0.4

OpenDAL introduces many breaking changes in v0.4.

Object::reader() is not AsyncSeek anymore

Since v0.4, Object::reader() will return impl BytesRead instead of Reader that implements AsyncRead and AsyncSeek. Users who want AsyncSeek please wrapped with opendal::io_util::seekable_read:

use opendal::io_util::seekable_read;

let o = op.object("test");
let mut r = seekable_read(&o, 10..);
r.seek(SeekFrom::Current(10)).await?;
let mut bs = vec![0;10];
r.read(&mut bs).await?;

Use RangeBounds instead

Since v0.4, the following APIs will be removed.

• Object::limited_reader(size: u64)
• Object::offset_reader(offset: u64)
• Object::range_reader(offset: u64, size: u64)

Instead, OpenDAL is providing a more general range_reader powered by RangeBounds:

pub async fn range_reader(&self, range: impl RangeBounds<u64>) -> Result<impl BytesRead>

Users can use their familiar rust range syntax:

let r = o.range_reader(1024..2048).await?;

Return io::Result instead

Since v0.4, all functions in OpenDAL will return std::io::Result instead.

Please check via std::io::ErrorKind directly:

use std::io::ErrorKind;

if let Err(e) = op.object("test_file").metadata().await {
    if e.kind() == ErrorKind::NotFound {
        println!("object not exist")
    }
}

Removing Credential

Since v0.4, Credential has been removed, please use the API provided by Builder directly.

builder.access_key_id("access_key_id");
builder.secret_access_key("secret_access_key");

Write returns BytesWriter instead

Since v0.4, Accessor::write will return a BytesWriter instead accepting a BoxedAsyncReader.

Along with this change, the old Writer has been replaced by a new set of write functions:

pub async fn write(&self, bs: impl AsRef<[u8]>) -> Result<()> {}
pub async fn writer(&self, size: u64) -> Result<impl BytesWrite> {}

Users can write into an object more easily:

let _ = op.object("path/to/file").write("Hello, World!").await?;

io_util replaces readers

Since v0.4, mod io_util will replace readers. In io_utils, OpenDAL provides helpful functions like:

• into_reader: Convert BytesStream into BytesRead
• into_sink: Convert BytesWrite into BytesSink
• into_stream: Convert BytesRead into BytesStream
• into_writer: Convert BytesSink into BytesWrite
• observe_read: Add callback for BytesReader
• observe_write: Add callback for BytesWrite

New type alias

For better naming, types that OpenDAL returns have been renamed:

• AsyncRead + Unpin + Send => BytesRead
• BoxedAsyncReader => BytesReader
• AsyncWrite + Unpin + Send => BytesWrite
• BoxedAsyncWriter => BytesWriter
• ObjectStream => ObjectStreamer

v0.3.0

What's Changed

• add azure blob support by @D2Lark in #165
• feat: Add tracing support via minitrace by @D2Lark in #175
• fix(service/s3): Endpoint template should be applied if region exists by @Xuanwo in #180
• chore: Level down some log entry to debug by @Xuanwo in #181
• feat(service/s3): Implement server side encryption support by @Xuanwo in #182
• Bump to version 0.3.0 by @Xuanwo in #186

New Contributors

• @D2Lark made their first contribution in #165

Full Changelog: v0.2.5...v0.3.0

v0.2.5

v0.2.5 - 2022-03-22

Added

• feat: Adopt quick_xml to parse xml (#164)
• test: Add behavior test for not exist object (#166)
• feat: Allow user input region (#168)

Changed

• feat: Improve error handle for s3 service (#169)
• feat: Read error response for better debugging (#170)
• examples: Improve examples for s3 (#171)

v0.2.4

v0.2.4 - 2022-03-18

Added

• feat: Add content_md5 and last_modified in metadata (#158)

Changed

• refactor: Say goodbye to aws-s3-sdk (#152)

v0.2.3

v0.2.3 - 2022-03-14

Added

• feat: Export BoxedObjectStream so that users can implement Layer (#147)

v0.2.2

v0.2.2 - 2022-03-14

Fixed

• services/fs: Refactor via tokio::fs (#142)
• fix: Stat root should return a dir object (#143)

v0.2.1

v0.2.1 - 2022-03-10

Added

• *: Implement logging support (#122)
• feat(service): Add service memory read support (#121)
• services: Add basic metrics (#127)
• services: Add full memory support (#134)

Changed

• benches: Refactor to support more read pattern (#126)
• services: Refactor into directories (#131)

Docs

• docs: Cover all public types and functions (#128)
• docs: Update README (#129)
• ci: Generate main docs to https://opendal.databend.rs (#132)
• docs: Enrich README (#133)
• Add examples for object (#135)