Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support for redis-cluster #23869

Closed
pat-s opened this issue Apr 2, 2023 · 13 comments · Fixed by #23975
Closed

Support for redis-cluster #23869

pat-s opened this issue Apr 2, 2023 · 13 comments · Fixed by #23975
Labels
Milestone

Comments

@pat-s
Copy link
Member

pat-s commented Apr 2, 2023

Feature Description

Using Gitea with redis-cluster seems not possible as of right now.
When attempting so, the following error is thrown by Gitea

gitea 2023/04/01 23:31:36 ...ers/web/auth/auth.go:312:handleSignInFull() [E] [6428bed8] RegenerateSession: regenerate session: CROSSSLOT Keys in request don't hash to the same slot

This is a unique behavior of redis-cluster compared to a single redis installation and must be accounted for on the client side as described in https://dzone.com/articles/resolved-crossslot-keys-error-in-redis-cluster-mod.

When creating keys that could be used by multi-key operations, use hashtags ({…}) to force the keys into the same hash slot. When the key contains a “{…}” pattern, only the substring between the braces, “{” and “}”, is considered to calculate the hash slot.
For example, the keys {user1}:mykey1 and {user1}:mykey2 are hashed to the same hash-slot because only the string inside the braces “{” and “}”, that is, “user1”, is used to compute the hash-slot.

I can't infer if this would be a simple addition or a breaking change but wanted to drop the information about redis-cluster anyhow as I couldn't find a prior discussion about it.

Screenshots

No response

@pat-s pat-s added type/proposal The new feature has not been accepted yet but needs to be discussed first. type/feature Completely new functionality. Can only be merged if feature freeze is not active. labels Apr 2, 2023
@lunny
Copy link
Member

lunny commented Apr 2, 2023

Could you paste your configuration and redis cluster information here?

@pat-s
Copy link
Member Author

pat-s commented Apr 2, 2023

I just used the default config of the bitnami chart. Important: I used the latest v6 redis (chart: release 7.6.4) as the v7 has some compatibility issues with the underlying Debian image and runs into another issue (unrelated to Gitea).

@wxiaoguang
Copy link
Contributor

wxiaoguang commented Apr 2, 2023

You can modify your redis conn string in your Gitea config , add a "prefix" with slot, then p.c.Rename(ctx, poldsid, psid) in code won't report the CROSSSLOT error. I guess: redis://..../?...&prefix={session}:

@lunny
Copy link
Member

lunny commented Apr 2, 2023

See the possible configuration connection string https://github.com/go-gitea/gitea/blob/main/modules/nosql/redis.go#L18-L22

@wxiaoguang
Copy link
Contributor

The "prefix" option is not documented, but cluster needs it. Make it prefix={session}: , then all sessions keys become {session}:random-session-id, it could make the redis cluster happy.

@silverwind
Copy link
Member

silverwind commented Apr 3, 2023

Tangentially related: There is also redis-sentinel, a kind of "proxy" daemon that may need special support inside our code. I think it can be used in addition to redis-cluster. It is the premier HA option for redis.

@wxiaoguang
Copy link
Contributor

To fix the problem, either:

  1. Let users do "?prefix={session}:" trick
  • OR -
  1. Do not use Rename in redis session provider , just Get and Set, then no cross-cluster problem.

@pat-s
Copy link
Member Author

pat-s commented Apr 7, 2023

I came across this while testing redis-cluster support for the helm chart. Happy to try again once there is a built-in fix for this which does not require manual modifications. This way we can have a "clean" start when adding support for redis-cluster.

@wxiaoguang
Copy link
Contributor

Can you try this PR: Use Get/Set instead of Rename when Regenerate session id #23975 ?

@pat-s
Copy link
Member Author

pat-s commented Apr 7, 2023

Not as a PR as I would need an image to test it with the Helm chart.

@wxiaoguang
Copy link
Contributor

Approve and merge, then get a dev image?

@lunny lunny added the type/bug label Apr 7, 2023
@lunny lunny added this to the 1.19.1 milestone Apr 7, 2023
@lunny lunny removed type/proposal The new feature has not been accepted yet but needs to be discussed first. type/feature Completely new functionality. Can only be merged if feature freeze is not active. labels Apr 7, 2023
@lunny
Copy link
Member

lunny commented Apr 7, 2023

It should be a bug rather than a feature because we have already supports redis cluster.

6543 pushed a commit that referenced this issue Apr 7, 2023
Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix #23869
GiteaBot pushed a commit to GiteaBot/gitea that referenced this issue Apr 7, 2023
)

Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix go-gitea#23869
jolheiser pushed a commit that referenced this issue Apr 7, 2023
…3983)

Backport #23975 by @wxiaoguang

Do not use Rename here, because the old sid and new sid may be in
different redis cluster slot.

Fix #23869

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
@pat-s
Copy link
Member Author

pat-s commented Apr 13, 2023

This PR together with #24114 resulted in a successful connection to a redis-cluster instance 🎉

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 29, 2023
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants