Introduce flags for fine-tuning maximum concurrent reconciles per resource #141
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a-hilaly
commented
Mar 2, 2024
Comment on lines
+209
to
+214
| flag.IntVar( | ||
| &cfg.ReconcileDefaultMaxConcurrency, flagReconcileDefaultMaxConcurrency, | ||
| 1, | ||
| "The default maximum number of concurrent reconciles for a resource reconciler. This value is used if no "+ | ||
| "resource-specific override has been specified. Default is 1.", | ||
| ) |
There was a problem hiding this comment.
We will maintain the default maximum reconciles at 1
a-hilaly
force-pushed
the
concurrent-reconciles
branch
4 times, most recently
from
a6f6167 to
ca3dc92
Compare
…ource. This commit aims to improve the performance and scalability of the ACK service controllers by introducing support for configuring the maximum number of concurrent reconciles for individual resources. The primary motivation behind this change is to address varying workload demands and resource requirements in Kubernetes environements. This patch introduces two new flags (soon exposed to in the helm chart values): - `--reconcile-default-max-concurrent-syncs`: allow users to specify the default maximum concurrency level for all the resources. - `--reconcile-resource-max-concurrent-syncs`: enable users to define resource-specific maximum concurrency settings, overriding the default value. A use case example would be the scenario of an admin wanting to manage EKS resources using the ACK `eks-controller`. It is normal for each cluster to have multiple nodegroups/PIAs/AccessEntries/FargateProfiles, which can indicate the proportion of needed max concurrencies for each resource. For instance, you can configure the EKS Controller with a default maximum of 2 concurrent reconciles for all the resources, and override the maximum concurrency to 10 for `AccessEntries` and `PodIdentityAssociations` Signed-off-by: Amine Hilaly <hilalyamine@gmail.com>
a-hilaly
force-pushed
the
concurrent-reconciles
branch
from
ca3dc92 to
8c58045
Compare
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: a-hilaly, jlbutler The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
ack-prow bot
pushed a commit
to aws-controllers-k8s/code-generator
that referenced
this pull request
Mar 5, 2024
Propagating two main patches to all the controllers: - Properly handle CARM errors and requeues by @a-hilaly in aws-controllers-k8s/runtime#140 - Introduce flags for fine-tuning maximum concurrent reconciles per resource by @a-hilaly in aws-controllers-k8s/runtime#141 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
ndbhat
pushed a commit
to ndbhat/ack-runtime
that referenced
this pull request
Apr 16, 2024
…ource (aws-controllers-k8s#141) This commit aims to improve the performance and scalability of the ACK service controllers by introducing support for configuring the maximum number of concurrent reconciles for individual resources. The primary motivation behind this change is to address varying workload demands and resource requirements in Kubernetes environements. This patch introduces two new flags (soon exposed to in the helm chart values): - `--reconcile-default-max-concurrent-syncs`: allow users to specify the default maximum concurrency level for all the resources. - `--reconcile-resource-max-concurrent-syncs`: enable users to define resource-specific maximum concurrency settings, overriding the default value. A use case example would be the scenario of an admin wanting to manage EKS resources using the ACK `eks-controller`. It is normal for each cluster to have multiple nodegroups/PIAs/AccessEntries/FargateProfiles, which can indicate the proportion of needed max concurrencies for each resource. For instance, you can configure the EKS Controller with a default maximum of 2 concurrent reconciles for all the resources, and override the maximum concurrency to 10 for `AccessEntries` and `PodIdentityAssociations` Signed-off-by: Amine Hilaly <hilalyamine@gmail.com> By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit aims to improve the performance and scalability of the ACK
service controllers by introducing support for configuring the maximum
number of concurrent reconciles for individual resources. The primary
motivation behind this change is to address varying workload demands and
resource requirements in Kubernetes environements.
This patch introduces two new flags (soon exposed to in the helm chart
values):
--reconcile-default-max-concurrent-syncs: allow users to specify thedefault maximum concurrency level for all the resources.
--reconcile-resource-max-concurrent-syncs: enable users to defineresource-specific maximum concurrency settings, overriding the default
value.
A use case example would be the scenario of an admin wanting to manage
EKS resources using the ACK
eks-controller. It is normal for eachcluster to have multiple nodegroups/PIAs/AccessEntries/FargateProfiles,
which can indicate the proportion of needed max concurrencies for each
resource.
For instance, you can configure the EKS Controller with a default
maximum of 2 concurrent reconciles for all the resources, and override
the maximum concurrency to 10 for
AccessEntriesandPodIdentityAssociationsSigned-off-by: Amine Hilaly hilalyamine@gmail.com
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.