Skip to content

Update the description about String 512MB limit #1653

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hayashier
Copy link
Contributor

@hayashier hayashier commented Sep 20, 2021

512 MB max value limit was removed. redis/redis#757
proto-max-bulk-len was introduced with Redis 4.0.7. This limit seems to have been removed at that time.

Actually I confirmed I can set the strings, the size of which is over 512MB by changing proto-max-bulk-len and client-query-buffer-limit

So I updated this description.

@itamarhaber
Copy link
Member

Hello @hayashier - thanks for this.

The 512MB limit has indeed been relaxed, but it is still the default behavior. To support bigger strings, one would first need to configure (proto-max-bulk-len directive) the server accordingly, so I'd rather have a clear explanation of that rather than deleting the relevant sentences.

Also, if you look through the docs, you'll find other references to the 512MB limit (e.g. SETBIT, SETRANGE and topics/protocol) so this would be a good chance to update all with this information.

@hayashier
Copy link
Contributor Author

hayashier commented Sep 20, 2021

Hello itamarhaber,

I agree. I found the following parts which describe 512MB limit by using grep command.
And I agree we should explain about proto-max-bulk-len directive. I'll think about the updated idea.

  1. https://redis.io/commands/client-setname

There is no limit to the length of the name that can be assigned if not the usual limits of the Redis string type (512 MB).

  1. https://redis.io/commands/setbit

The offset argument is required to be greater than or equal to 0, and smaller than 232 (this limits bitmaps to 512MB). When the string at key is grown, added bits are set to 0.

On a 2010 MacBook Pro, setting bit number 232 -1 (512MB allocation) takes ~300ms, setting bit number 230 -1 (128MB allocation) takes ~80ms, setting bit number 228 -1 (32MB allocation) takes ~30ms and setting bit number 226 -1 (8MB allocation) takes ~8ms.

  1. https://redis.io/commands/setrange

Note that the maximum offset that you can set is 229 -1 (536870911), as Redis Strings are limited to 512 megabytes. If you need to grow beyond this size, you can use multiple keys.

On a 2010 MacBook Pro, setting byte number 536870911 (512MB allocation) takes ~300ms, setting byte number 134217728 (128MB allocation) takes ~80ms, setting bit number 33554432 (32MB allocation) takes ~30ms and setting bit number 8388608 (8MB allocation) takes ~8ms.

  1. https://redis.io/topics/data-types

A String value can be at max 512 Megabytes in length.

  1. https://redis.io/topics/modules-api-ref

The function returns REDISMODULE_OK on success, and REDISMODULE_ERR on error, that is, the key is not open for writing, is not a string or resizing for more than 512 MB is requested.

  1. https://redis.io/topics/protocol

Bulk Strings are used in order to represent a single binary safe string up to 512 MB in length.

@hayashier hayashier changed the title Remove the description about String 512MB limit Update the description about String 512MB limit Sep 20, 2021
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Shogo Hayashi seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants