data.template_file._environment_keys: value of 'count' cannot be computed

[colindean]

Or, more specifically to my case:

module.ecs_cluster.module.agent_mgmt_ecs.module.container_def.data.template_file._environment_keys: data.template_file._environment_keys: value of 'count' cannot be computed

I'm inside a module that is defining and passing a map into a module that is then passing that map into terraform_container_definitions. When I remove one of the environment variables I'm setting, it works. The trick is that that value is computed: I'm passing AWS RDB connection info that I'm defining within the same module (ecs_cluster, in the same file where I'm defining agent_mgmt_ecs, wherein I'm creating this environment variable map.

The error is here: https://github.com/Eiara/terraform_container_definitions/blob/master/main.tf#L57:

data "template_file" "_environment_keys" {
  count = "${length(keys(var.environment))}"

  template = <<JSON
{
  "name": $${name},
  "value":$${value}
}
JSON

  vars {
    name  = "${jsonencode( element(keys(var.environment), count.index) )}"
    value = "${jsonencode( lookup(var.environment, element(keys(var.environment), count.index)) )}"
  }
}

I think that Terraform is having trouble getting the keys on a map that has a computed value for one of the keys. I'm thinking that maybe I need a depends_on somewhere, but I'm having trouble tracking down where. terraform_container_definitions itself does not take a depends_on so I'd have to insert it somewhere else…

I could be "just doing it wrong" and would appreciate a shove in a direction, if you can provide that. There's minimally something to be added to terraform_container_definitions saying that one cannot do what I'm trying to do, if that's the case.

[colindean]

I came up with this with the same file as where I'm creating the database and passing the environment variable map into the module where terraform_container_definitions uses said map:

data "template_file" "db_url" {
  template = "$${engine}://$${username}:$${password}@$${endpoint}/$${database}"
  vars = {
    engine = "${local.database_engine}"
    username = "${module.agent_mgmt_db.this_db_instance_username}"
    password = "${module.agent_mgmt_db.this_db_instance_password}"
    endpoint = "${module.agent_mgmt_db.this_db_instance_endpoint}"
    database = "${module.agent_mgmt_db.this_db_instance_name}"
  }
  depends_on = ["module.agent_mgmt_db"]
}

then I realized that this template should already know that it needs to depend on module.agent_mgmt_db implicitly.

If it helps, the module creating the database instance is the official rds module in the registry: (repo) (registry).

It should be noted that if I comment out the endpoint line within the environment map, terraform plan works and I can apply. If uncomment the line after doing so, everything resolves correctly. So, I think it's a dependency problem.

[colindean]

The error is happening with not just the database module output, either. In my environment map, I added – after a successful run without any dependencies – this keyvalue pair:

DOWNLOAD_SERVICE_URL = "${module.download_ecs.service_fqdn}"

That module is an instance of an internal module that just wraps around the ECS resources to create a Fargate module.

So, I'm reaching a conclusion that terraform_container_definitions in its current state cannot handle computed values passed into it, at least for environment variables but probably for everything since all properties seem to be handled similarly.

[colindean]

I tried switching length(keys(var.environment)) to length(values(var.environment)). Looking through the interpolation syntax, I don't see another way to effectively count the number of elements in a map in Terraform.

[colindean]

Found a relevant issue: hashicorp/terraform#15471

And the master issue asking for documentation about it: hashicorp/terraform#17421

It's suggested here that passing the count explicitly may be possible. I'm going to try a modification with that…

[colindean]

I tried adding a separate environment_computed structure to the module. The outcome was that the static values were present in the output but the computed ones were not on the first apply.

diff --git a/main.tf b/main.tf
index e114b96..9b0f88a 100644
--- a/main.tf
+++ b/main.tf
@@ -79,6 +79,39 @@ JSON
   }
 }

+# Constructs the environment K/V from a map with computed values
+# This exists because of https://github.com/hashicorp/terraform/issues/17421
+# Use this when an environment contains computed values and set "environment_computed_count" as well.
+# If you don't use this and try to pass computed values to "environment" then you will get this error:
+#     data.template_file._environment_keys: value of 'count' cannot be computed
+# Prevents an envar from being declared more than once, as is sensible
+
+data "template_file" "_environment_keys_computed" {
+  count = "${var.environment_computed_count}"
+
+  template = <<JSON
+{
+  "name": $${name},
+  "value":$${value}
+}
+JSON
+
+  vars {
+    name  = "${jsonencode( element(keys(var.environment_computed), count.index) )}"
+    value = "${jsonencode( lookup(var.environment_computed, element(keys(var.environment_computed), count.index)) )}"
+  }
+}
+
+data "template_file" "_environment_list_computed" {
+  template = <<JSON
+  "environment": [$${environment}]
+JSON
+
+  vars {
+    environment = "${join(",",data.template_file._environment_keys_computed.*.rendered)}"
+  }
+}
+
 # Done this way because of module boundaries casting booleans to 0 and 1

 data "template_file" "_mount_keys" {
@@ -206,6 +238,7 @@ JSON
           "${length(var.links) > 0 ? "${jsonencode("links")}: ${jsonencode(var.links)}" : ""}",
           "${length(var.port_mappings) > 0 ?  data.template_file._port_mappings.rendered : ""}",
           "${length(keys(var.environment)) > 0 ? data.template_file._environment_list.rendered : "" }",
+          "${length(keys(var.environment_computed)) > 0 ? data.template_file._environment_list_computed.rendered : "" }",
           "${length(var.mount_points) > 0 ? data.template_file._mount_list.rendered : "" }",
           "${length(var.volumes_from) > 0 ? data.template_file._volumes_from_list.rendered : "" }",
           "${length(var.command) > 0 ? "${jsonencode("command")}: ${jsonencode(var.command)}" : "" }",

Applying twice will kinda suck so I'm going to search for another workaround.

[colindean]

What I ended up doing is something like this, where I'm capturing the output of the module, having set some of the environment variables to templatable value. I'm then feeding the JSON output through a data.template_file and passing the real values in through vars in that template application.

module "container_def" {
  source = "git::https://github.com/colindean/terraform_container_definitions.git?ref=patch-1" 
  # source = "github.com/eiara/terraform_container_definitions"

  name        = "foobar"
  image       = "foobar"
  essential   = true
  memory      = "512"
  cpu         = "512"
  environment = {
    # no computed values; computed values must be templated and set for real in
    # `data.template_file.container_def` below.
    DATABASE_HOST = "$${database_host}"
    DATABASE_PORT     = "${module.agent_mgmt_db.this_db_instance_port}"
    DATABASE_USER     = "${module.agent_mgmt_db.this_db_instance_username}"
    DATABASE_PASSWORD = "${module.agent_mgmt_db.this_db_instance_password}"
    DATABASE_NAME     = "${module.agent_mgmt_db.this_db_instance_name}"
    DATABASE_URL = "$${database_url}"
    DOWNLOAD_SERVICE_URL = "$${download_service_url}"
}
  port_mappings = [{
    container_port = "${var.container_port}"
    host_port      = "${var.host_port}"
  }]
  logging_driver = "awslogs"
  logging_options = {
    awslogs-group         = "${var.log_group}"
    awslogs-region        = "${var.aws_region}"
    awslogs-stream-prefix = "${var.name}"
  }
}
data "template_file" "container_def" {
  template = "${module.container_def.json}"
  vars     = {
    # these are all computed values
    # anything in here that's not safe in JSON must be `jsonencode()`'d.
    database_host        = "${module.agent_mgmt_db.this_db_instance_address}"
    database_url         = "${data.template_file.db_url.rendered}"
    download_service_url = "${module.download_ecs.service_fqdn}"
  }
}

This seems to be working sufficiently for me.

[colindean]

I just referenced this in a chat conversation. I'll mention that my example above was a hair off: I ended up having to move the module.agent_mgmt_db.* envvars into the template_file vars. They're calculated, too, but they already existed when I was working on this code snippet. I discovered their true nature when I next did a destroy/apply cycle.

[zentavr]

@colindean - I wonder if you were able to resolve the issue?

[colindean]

No. What I have above is how it is, or at least was: I no longer have access to that codebase.