Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

rawx: Misc. improvements #2028

Merged
merged 9 commits into from
May 13, 2020
Merged

rawx: Misc. improvements #2028

merged 9 commits into from
May 13, 2020

Conversation

jfsmig
Copy link
Contributor

@jfsmig jfsmig commented May 5, 2020

SUMMARY
xattr

Make the rawx interleave the bunch of fsexattr with the bunch of write. In facts, the xattr are set before the final block (whose size is configurable, default=2MiB). On small chunks, i.e. smaller than the block size, the extended attributes will be stored before the data, so that a GET on the chunk does all the I/O operations sequentially. Prior to this change, the xattr where stored after the data and a GET would read the end of the file and then jump backwards, to the beginning. This cannot be efficient on HDD.

access logs

When logs are not necessary, it is interesting to stop logging to relax the log management chain (especially because the rawx is one of the worst emitters). So, 3 variables have been added to turn the logs on/off for specific chunk requests:

  • log_access_get: GET, HEAD (enabled by default)
  • log_access_put: PUT only (enabled by default)
  • log_access_delete: DELETE only (enabled by default)

Also, the emission of access log lines are not blocking anymore thanks to a go channel. If the emission would have blocked, an alert is raised and the line is wasted.

beanstalkd

We rewrite the connectors to have 1 queue / N workers, so that we don't block anymore on the connection to one backend. We monitor the availability of the queue and write an alert if the queue would have blocked the caller. Upon a full queue, the event is written in the log.

socket options

Allow to configure specific TCP options: tcp_cork (disabled by default) and tcp_nodelay (disabled by default).

ISSUE TYPE

enhancement

COMPONENT NAME

rawx

SDS VERSION
openio 5.4.1.dev2

@jfsmig jfsmig changed the title rawx: Write the xattr before the last block rawx: Misc. improvements May 13, 2020
@jfsmig jfsmig requested review from AymericDu and fvennetier May 13, 2020 09:58
@jfsmig jfsmig merged commit 69bf19e into open-io:5.x May 13, 2020
@jfsmig jfsmig deleted the 5.x-rawx-pre-xattr branch August 5, 2020 08:16
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants