How to dynamically configure the Tenants definitions with an API?

[Dentrax]

Hey, I was thinking about to configure the some Tenant.spec. values using an external API to fetch corresponding values of the desired resources.

Imagine I developed an QuotaAPI that running in the cluster as a service. It returns some bulk data in JSON format:

[
 "tenant-1": {
  "hard": {
    "cpu": 2,
    "mem": 2
  }
 },
 "tenant-2": {
  "hard": {
    "cpu": 3,
    "mem": 3
  }
 }
]

Use-case: this API is needed because cluster has auto-scale enabled. All values is calculated automatically according to resources we have.

For example, I'd like to set resourceQuotas and limitRanges for each Tenant differently by using my QuotaAPI.

Obviously, this would break Capsule's declarative manner, but I can not see any problem in imperative way.

Q/As:

• How can I do that?
  • Should I need to create an external operator that upsert those values in Tenant resource?
  • If so, if I update the Tenant resource, will Capsule catch the change and update the resources that Capsule created? (LimitRanges, Quotas, etc.)
• How Capsule will behave if namespace's hard limit CPU/MEM quota is 2, usage is 1.5, and API try to set to lower than 1.5?
• Does it make sense in overall?

[prometherion]

• Should I need to create an external operator that upsert those values in Tenant resource?

Honestly, I have no idea. An external operator could be good, although this could be done also by KEDA but I'm worried you have to write your own scaler (the one that would update the Tenant)

• If so, if I update the Tenant resource, will Capsule catch the change and update the resources that Capsule created? (LimitRanges, Quotas, etc.)

Absolutely, all the changes applied to the Tenant definition are reflected in all the children resources, ResourceQuota and LimitRange for the given case.

• How Capsule will behave if namespace's hard limit CPU/MEM quota is 2, usage is 1.5, and API try to set to lower than 1.5?

The Tenant resource quota management is delegated to the dynamic admission controller ResourceQuota: we're reusing its API to avoid adding additional machinery and reuse the Kubernetes API primitives. If you're downsizing the quota compared to the current usage, no further resources can be created. I'd say you're not going to expect any side effect.

• Does it make sense in overall?

I know @MaxFedotov implemented at WarGaming their own auto scaler, maybe his perspective can help you nail your goal.