Annotation vault.hashicorp.com/auth-config values are overridden

[mrjoelkamp]

Describe the bug
Both of the following annotations get overridden by code when specifying the role and token-path using the vault.hashicorp.com/auth-config annotation:
vault.hashicorp.com/auth-config-token-path
vault.hashicorp.com/auth-config-role

vault-k8s/agent-inject/agent/annotations.go

Lines 865 to 879 in 4f81d9d

	prefix := fmt.Sprintf("%s-", AnnotationVaultAuthConfig)
	for annotation, value := range a.Annotations {
	if strings.HasPrefix(annotation, prefix) {
	param := strings.TrimPrefix(annotation, prefix)
	param = strings.ReplaceAll(param, "-", "_")
	authConfig[param] = value
	}
	}
	if a.Vault.Role != "" {
	authConfig["role"] = a.Vault.Role
	}
	if a.ServiceAccountTokenVolume.MountPath != "" && a.ServiceAccountTokenVolume.TokenPath != "" {
	authConfig["token_path"] = path.Join(a.ServiceAccountTokenVolume.MountPath, a.ServiceAccountTokenVolume.TokenPath)
	}

Specifically, when more than one ServiceAccount volume is mounted to the pod and the agent.go code selects the first volume mount containing the string serviceaccount.

vault-k8s/agent-inject/agent/agent.go

Lines 767 to 777 in 4f81d9d

	for _, container := range pod.Spec.Containers {
	for _, volumes := range container.VolumeMounts {
	if strings.Contains(volumes.MountPath, "serviceaccount") {
	return &ServiceAccountTokenVolume{
	Name: volumes.Name,
	MountPath: volumes.MountPath,
	TokenPath: "token",
	}, nil
	}
	}
	}

Need the ability to explicitly set the correct token_path.

To Reproduce
Steps to reproduce the behavior:

1. Deploy application annotated for vault-agent injection that includes:

vault.hashicorp.com/auth-config-token-path: /path/to/some/token

2. Inspect the resulting VAULT_CONFIG and see that the token_path is not assigned to the specified path but the mounted service account volume path when a Service Account volume mount exists

Expected behavior
When using the vault.hashicorp.com/auth-config annotation, I expect that the values are not overridden.

Environment

• Kubernetes version:
  • EKS 1.22
• vault-k8s version:
  • 1.12.1

[tvoran]

Hi @mrjoelkamp, can you include all the annotations that would be used in your example application? I think I understand the issue, but want to make sure I have the full picture.

[mrjoelkamp]

Hi @mrjoelkamp, can you include all the annotations that would be used in your example application? I think I understand the issue, but want to make sure I have the full picture.

@tvoran yes, here is an example of all the annotations for an example app using this:

    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: example-app-kubernetes-ro
    vault.hashicorp.com/agent-run-as-user: "1337"
    vault.hashicorp.com/agent-run-as-group: "1337"
    vault.hashicorp.com/agent-inject-secret-config.toml: secret_v2/example/app/secret
    vault.hashicorp.com/agent-inject-template-config.toml: |
      [example]
      {{- with secret "secret_v2/example/app/secret" }}
      example_secret = "{{ .Data.data.example_secret }}"
      {{- end }}
    vault.hashicorp.com/auth-config-token-path: /var/run/secrets/kubernetes.io/serviceaccount

When inspecting the resulting VAULT_CONFIG after the pod is configured with Vault Agent Injector the token_path is set to /var/run/secrets/eks.amazonaws.com/serviceaccount in the case of attaching an IRSA role to the ServiceAccount. I would expect the vault.hashicorp.com/auth-config-token-path: annotation to set token_path to the value I provided /var/run/secrets/kubernetes.io/serviceaccount.

This causes the Vault agent authentication to fail because it isn't using the correct ServiceAccount token.

[tvoran]

Gotcha, yeah I think this is because serviceaccount() is returning the first volume it finds with serviceaccount in the name. Have you tried the agent-service-account-token-volume-name annotation as a workaround?

[mrjoelkamp]

Gotcha, yeah I think this is because serviceaccount() is returning the first volume it finds with serviceaccount in the name. Have you tried the agent-service-account-token-volume-name annotation as a workaround?

Yes exactly, I wrote that it selects the first volume mount it finds with a code snippet in the summary of the issue.

I've used the vault.hashicorp.com/agent-service-account-token-volume-name annotation as a workaround but it is very cumbersome because it requires adding a static named volume in an addition to the dynamically generated volume that gets added to the pod via webhooks. So, in order to use this as a workaround I have to add a new volume and volumeMount with a deterministic name. This ends up being a lot of extra cruft for what would otherwise be a one-line annotation pointing to the correct token path.

[tvoran]

Fixed in #457