-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Ability to delay scaling after deployment changes #1386
Comments
Hi, sorry I don't follow what is your problem. Could you please give us more details?
RabbitMQ queues defined where? |
There is one queue per application, it changes rarely. The problem is with exchanges, because they configured when the application starts (basing on registered implementations of IConsumer). Example: https://masstransit-project.com/usage/consumers.html#consumer. |
I still don't get it sorry 🤷♂️ So my understanding is:
|
Queue can be bound to many exchanges so an application can consume many types of events from one queue.
There is missing configuration step in point 3, because configuration of exchanges bound to the queue is done in runtime. There is no runtime, because keda scaled deployment to 0. The problem is with external dependencies that are configured in runtime by the application, but it cannot occur because the deployment has 0 pods. KEDA maintains 0 pods, but do not know that metric value could be higher if the application had a chance to configure those external dependencies. |
Maybe KEDA is not able to correctly decide if it should scale due to misconfiguration, there are a few options:
|
okay, gotcha. I am still not sure what could KEDA do about this as this is something very specific. You can temporarily remove the ScaledObject, so the Deployment will be scaled to default number of replicas, let it do the configuration changes and the re-create SO, or you can set the |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions. |
This issue has been automatically closed due to inactivity. |
Hi,
We are facing some issues/problems with integrating our applications with keda.
Background:
We have .net based applications with MassTransit/RabbitMQ. When an instance of application starts it configures exchanges and queues (adds proper exchanges mostly).
Example:
The application X is deployed on kubernetes (with keda scaled object). Keda scales the application within 0-3 range. The application listens to event A.
One day we changes the application and now it starts to listen to event B. We deploy the application on kubernetes cluster. Number of pods is 0. What happens? RabbitMQ queues and exchanges are still configured as it was before our changes. Event B never reaches our application, because there are only exchanges for event A (which will never come again for some reason).
No scaling occurs, no pod with newer images starts.
What can we do with that?
We thought about an ability to delay scaling after the deployment changed - maybe some kind of readiness check for keda? Or simple interval of time to await.
Or maybe the solution is ready 😃 and you can help us without code changes.
The text was updated successfully, but these errors were encountered: