release-2.2.25 with libsodium #17

LuckyFellow · 2016-08-01T07:57:49Z

No description provided.

The default max_buffer_size=256 was wrong in all situations. We're now assuming that the underlying istreams' max_buffer_size is always correct. While gluing together two streams we're now allocating enough memory to hold all of the wanted data (instead of assert-crashing as could have happened earlier). This means that the max memory usage is actually the two streams' max_buffer_size summed together. Ideally this would be fixed to limit the max_buffer_size to maximum of the two, but that would require further changes.

It's not enough to have the buffer size set to "large enough" at the time of the stream creation, because i_stream_set_max_buffer_size() could be called afterwards.

We were casting the wrong stream.

Make sure we don't grow the buffer size then.

…een decode buffer and stream buffer. Instead, use the decode buffer directly as stream buffer.

We can't handle an abort after an async fs_write_stream_finish() is already going.

The caller already set success==FALSE if ostream was closed.

As mentioned in a0cf7d3, this can't happen.

I had this code at some point earlier, but looks like it got lost from the final commit.

This fixes i_stream_get_max_buffer_size() to work correctly with istream-chain.

After 7be8ba0 it's no longer correct to access stream->max_buffer_size directly. These changes fix using istream-chain with various wrapper streams.

For example: - lookup start - iterate start - iterate finished, but can't reply yet - lookup finished - iterate reply can be sent now, but wasn't previously

All files incluided in dist are explicitly mentioned. The whole subdirectory 'stopwords' could also be distributed, but that is more error prone.

The reading tries to be a little bit stricter now. Only stopwords at the start of a new line are accepted now. Changed fi stopwords accordingly. Also removed superfluous stack allocation in parsing.

…nection while a request payload was still being read.

This changes the segfault of `doveadm -f formatted ...` to an error.

These correspond to IMAP BODY[<section>].

This was done for mail commands while initializing the mail user, but other commands weren't using it. This meant that doveadm was using only global settings instead of protocol doveadm { .. } settings for everything except mail commands.

…ate. This continues 3d49dc6, which didn't actually work because brain->require_full_resync was either cleared earlier or it was never even set in this brain.

…late" for now It always happens at least with Ubuntu 16.04.

We should stop while waiting for anvil reply. Broken by ced943b.

This caused stopping to be too slow. Broken by 0153cf5.

…nection_server_close(). Used wrong variable: it is set to NULL by http_client_request_unref() before it is used. Problem found by Coverity.

Potentially reduces latency at the end of a larger data transfer.

The check was done too late after recent changes.

This fixes a race condition where the http_client_host_shared_idle_timeout() function would get called with an already freed hshared argument. Specifically, the situation arises from the hshared idle timeout calling http_client_host_shared_free(), which removes the timeout and then proceeds to free the client queue. The client queue freeing code indirectly calls http_client_host_shared_check_idle(), which notices that there is no idle timeout and allocates one. The backtrace at the point of this new timeout allocation: frame #3: 0x00007f0c775897f0 libdovecot.so.0`timeout_add_to(...) ioloop.c:280 frame #4: 0x00007f0c7751a45f libdovecot.so.0`http_client_host_shared_check_idle(hshared=<unavailable>) at http-client-host.c:69 frame #5: 0x00007f0c7750de89 libdovecot.so.0`http_client_request_error(_req=<unavailable>, status=9000, error="") at http-client-request.c:1525 frame #6: 0x00007f0c77517f38 libdovecot.so.0`http_client_queue_fail_full(queue=0x000055e13cff0e10, status=9000, error="", all=<unavailable>) at http-client-queue.c:183 frame #7: 0x00007f0c77518baa libdovecot.so.0`http_client_queue_free(queue=0x000055e13cff0e10) at http-client-queue.c:141 frame #8: 0x00007f0c7751a8bc libdovecot.so.0`http_client_host_free_shared(_host=<unavailable>) at http-client-host.c:391 frame #9: 0x00007f0c7751ab4c libdovecot.so.0`http_client_host_shared_free(_hshared=0x00007ffdac109e48) at http-client-host.c:294 frame #10: 0x00007f0c7751ace8 libdovecot.so.0`http_client_host_shared_idle_timeout(hshared=<unavailable>) at http-client-host.c:40 frame #11: 0x00007f0c7758a1a4 libdovecot.so.0`io_loop_handle_timeouts at ioloop.c:682 frame #12: 0x00007f0c7758a089 libdovecot.so.0`io_loop_handle_timeouts(ioloop=0x000055e13cfc8d80) at ioloop.c:696 frame #13: 0x00007f0c7758befc libdovecot.so.0`io_loop_handler_run_internal(ioloop=0x000055e13cfc8d80) at ioloop-select.c:126 frame #14: 0x00007f0c7758a56d libdovecot.so.0`io_loop_handler_run(ioloop=<unavailable>) at ioloop.c:767 frame #15: 0x00007f0c7758a798 libdovecot.so.0`io_loop_run(ioloop=0x000055e13cfc8d80) at ioloop.c:740 frame #16: 0x00007f0c774f61eb libdovecot.so.0`master_service_run(service=0x000055e13cfc8c10, callback=<unavailable>) at master-service.c:782 frame #17: 0x000055e13b48e3a5 stats`main(argc=<unavailable>, argv=<unavailable>) at main.c:99 frame #18: 0x00007f0c771092e1 libc.so.6`__libc_start_main + 241 frame #19: 0x000055e13b48e41a stats`_start + 42

sirainen and others added 30 commits June 1, 2016 12:08

lib-dcrypt: Fixed check program dependencies

eabf965

doveadm: Do not use INT64 with mail cmds yet

936f81c

fs-metawrap: Fixed buffer size while reading metadata header.

9e7b4d6

It's not enough to have the buffer size set to "large enough" at the time of the stream creation, because i_stream_set_max_buffer_size() could be called afterwards.

lib: Fix to previous istream-chain max_buffer_size handling

3dafbab

We were casting the wrong stream.

lib: istream's max_buffer_size=0 means 0, not unlimited.

5b6559c

Make sure we don't grow the buffer size then.

lib-mail: istream-qp-decoder: For efficiency, avoid copying data betw…

a69f793

…een decode buffer and stream buffer. Instead, use the decode buffer directly as stream buffer.

lib-fs: Added fs_write_stream_abort_async()

5f4111f

lib-fs: Clarified fs_write_stream_abort() API and dropped its _async().

d010574

We can't handle an abort after an async fs_write_stream_finish() is already going.

lib-fs: Added more asserts

90ec94c

fs-metawrap: Removed unnecessary code.

f412ee7

The caller already set success==FALSE if ostream was closed.

lib-fs: Improved unit tests.

1d29d49

fs-metawrap: Removed unnecessary code.

3ffc655

As mentioned in a0cf7d3, this can't happen.

lib-compression: istream.max_buffer_size == 0 isn't unlimited.

6bd205f

lib: istream-chain didn't update its max_buffer_size

51dcb33

I had this code at some point earlier, but looks like it got lost from the final commit.

lib: i_stream_get_max_buffer_size() checks also parents' max sizes

41a18c1

This fixes i_stream_get_max_buffer_size() to work correctly with istream-chain.

global: Use i_stream_get_max_buffer_size() wherever possible

f67f6de

After 7be8ba0 it's no longer correct to access stream->max_buffer_size directly. These changes fix using istream-chain with various wrapper streams.

dict: Fixed crash when reaching max (5) pending commands.

72f0027

dict: Pipelined iteration replies may have been hanging.

8b895d0

For example: - lookup start - iterate start - iterate finished, but can't reply yet - lookup finished - iterate reply can be sent now, but wasn't previously

lib-fts: Move stopwords to subdirectory.

c7021f4

All files incluided in dist are explicitly mentioned. The whole subdirectory 'stopwords' could also be distributed, but that is more error prone.

lib-fts: Add stopword files for more languages.

77c9af3

lib-fts: Improved stopword file reading.

f6e1440

The reading tries to be a little bit stricter now. Only stopwords at the start of a new line are accepted now. Changed fi stopwords accordingly. Also removed superfluous stack allocation in parsing.

auth: Add PBKDF2 password scheme

b89dd50

lib: Add PKCS#5 pbkdf1 and 2

71e8344

lib-dict-extra: Avoid out of scope data stack usage.

a5658a1

doveadm-server: Handle istream send error.

0fbf2c9

.gitignore: Added *~

5c33861

lib-http: server: Fixed assert failure occurring when closing the con…

273ed7a

…nection while a request payload was still being read.

doveadm: error to print formatted without format

2812b0e

This changes the segfault of `doveadm -f formatted ...` to an error.

doveadm fetch: Added body.<section> and binary.<section> fields

7ec76ff

These correspond to IMAP BODY[<section>].

sirainen and others added 25 commits June 29, 2016 19:24

dict-sql: Treat NULL value the same as "key not found"

923ed58

lib: iostream-temp: Fixed o_stream_send_istream() with >2GB files

8051312

doveadm user: Avoid potential crashes when running via doveadm-server

47441da

doveadm: Read settings with service=doveadm

06aeecf

This was done for mail commands while initializing the mail user, but other commands weren't using it. This meant that doveadm was using only global settings instead of protocol doveadm { .. } settings for everything except mail commands.

dsync: When full resync is wanted in a stateful sync, output empty st…

7be1766

…ate. This continues 3d49dc6, which didn't actually work because brain->require_full_resync was either cleared earlier or it was never even set in this brain.

auth: Fixed checking if delay_until is too large

9892356

auth: delay_until can optionally now have +<max random secs> suffix.

89ed1b1

lib-ssl-iostream: Disable "CRYPTO_set_mem_functions() was called too …

ec56d6d

…late" for now It always happens at least with Ubuntu 16.04.

lib-ssl-iostream: Compiler fix to previous commit

2272c47

dcrypt: Fix bugs in 0 and 1 byte payload files

822ecda

dcrypt: Add tests for 0 and 1 byte data

9f46e5f

dcrypt: Free keys in test-stream

1be434c

auth-request: Correctly allocate mech_password and credentials_scheme

3a1d14c

auth-request: Whitespace fix

4fd02ab

lmtp: Fixed pipelining of commands after RCPT TO

129abc3

We should stop while waiting for anvil reply. Broken by ced943b.

lmtp: Removed assert, which triggers if anvil connect fails.

dbca72d

Released v2.2.25.rc1.

1969dd8

master: Stopping didn't close dead-pipes early enough.

3c432ac

This caused stopping to be too slow. Broken by 0153cf5.

imap: Fixed reading LDA settings from config.

abd85e9

lib-dcrypt: Compiling fix

88a6c0d

lib-http: client: Fixed potential segfault problem in http_client_con…

975f69e

…nection_server_close(). Used wrong variable: it is set to NULL by http_client_request_unref() before it is used. Problem found by Coverity.

configure: Check if C99 "static" keyword is supported for array sizes.

522bb22

login-proxy: cork+uncork in flush callbacks

ad7302c

Potentially reduces latency at the end of a larger data transfer.

doveadm mailbox update: Avoid assert-crash on errors.

8d3f4a6

lib-storage: Autoexpunging should ignore nonexistent mailboxes.

b2f9b1d

The check was done too late after recent changes.

LuckyFellow changed the title ~~release-2.2.25with libsodium~~ release-2.2.25 with libsodium Aug 1, 2016

LuckyFellow closed this Aug 1, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

release-2.2.25 with libsodium #17

release-2.2.25 with libsodium #17

LuckyFellow commented Aug 1, 2016

release-2.2.25 with libsodium #17

release-2.2.25 with libsodium #17

Conversation

LuckyFellow commented Aug 1, 2016