[Feature] Redeploy/destroy of Machine Catalog should put machines into maintenance mode automatically

[anderswagner]

Describe the feature request

Fill out the following questions and add any additional information.
Summary:

For the citrix_machine_catalog, when it is trying to destroy the machine catalog (either for a replacement or simply destroying it), it will fail due to the following error

│ Error: Error deleting Machine Catalog ABC
│ 
│ TransactionId: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx
│ JobId: xxxxxxxx-xxxx-xxxxx-xxxxx-xxxxxxxxxxxxx
│ Error message: Unable to delete Machine Catalog because: 4 Machines are not
│ in maintenance mode
╵

The provider should be able to log off the users, power off the machines and put them into maintenance mode automatically for the delete to go through. These options are currently not available for the consumer of this provider to code into their terraform code, or for it to be done automatically in this version, therefore this request.

Link to any docs that cover this feature:
https://docs.citrix.com/en-us/xenapp-and-xendesktop/7-15-ltsr/install-configure/machine-catalogs-manage.html#delete-machines-from-a-machine-catalog

Terraform code for this resource for clarity:

resource "citrix_machine_catalog" "default" {
  name                        = var.catalog.publishedName
  allocation_type             = var.catalog.allocationType
  zone                        = data.citrix_zone.azure_location_zone.id
  machine_catalog_folder_path = "omitted"

  provisioning_type = "MCS"
  session_support   = "SingleSession"
  scopes            = ["00000000-0000-0000-0000-000000000000"]

  persist_user_changes     = var.catalog.persistUserChanges != null ? var.catalog.persistUserChanges : "Discard"
  minimum_functional_level = var.catalog.functionalLevel

  provisioning_scheme = {
    hypervisor               = data.citrix_hypervisor.azure_hypervisor.id
    hypervisor_resource_pool = citrix_azure_hypervisor_resource_pool.default.id
    identity_type            = "ActiveDirectory"
    number_of_total_machines = var.catalog.numberOfMachines

    machine_domain_identity = {
      domain                   = var.domainInfo.domain
      service_account          = var.domainInfo.domainAdminUsername
      service_account_password = var.domainInfo.domainAdminPassword

      domain_ou = var.catalog.ou != null ? var.catalog.ou : local.default_ou
    }

    network_mapping = [{
      network        = var.catalog.subnetName
      network_device = "0"
    }]

    azure_machine_config = {
      storage_type      = "Premium_LRS"
      use_managed_disks = true
      license_type      = "Windows_Client"

      service_offering   = var.catalog.vmSize
      vda_resource_group = var.vdaResourcegroup
      master_image_note  = var.imageVersion

      azure_master_image = {
        master_image   = var.catalog.managedDisk
        resource_group = var.catalogResourceGroup
      }
    }

    machine_account_creation_rules = {
      naming_scheme = lower(
        "omitted"
      )
      naming_scheme_type = "Numeric"
    }
  }
}

[xushengl]

Hello @anderswagner

Thanks for reporting this issue, we will work on a fix for this issue.

Best Regards,
Xusheng "Fred" Liu

[aneeshk-citrix]

@anderswagner Are these machines in the machine catalog a part of some delivery group that is not managed by terraform?

[anderswagner]

@aneeshk-citrix For this particular instance it was machine catalog, delivery group etc. provisioned manually through the API. Then both machine catalog and delivery group was imported into terraform and when trying to run terraform plan + apply it wanted to replace the machine catalog due to one of the properties changing, and then it fails.

[aneeshk-citrix]

@anderswagner A couple of things to discuss here:

1. Do you expect a replace operation on the machine catalog after importing?
   From the config, I see you are specifying persist_user_changes. Is this triggering the replace?
2. The error you see when deleting machine catalog - machines not being in maintenance mode, happens because machines are still a part of some delivery group. We will make a fix to perform some validation and ensure that machines in machine catalog being deleted should not be a part of any delivery group.

Can you also post the config for delivery group? I'm assuming you do not want a resource dependency between delivery group and machine catalog. (With resource dependency, if machine catalog is being deleted, it will delete the delivery group first. This might be a workaround for your issue for now).

Ideally, there shouldn't be a replace on a resource right after importing it (unless, of course, you intend to change a nontrivial property that needs resource replacement).

Based on your replies, we can plan for the appropriate fixes and I can also suggest some workarounds till we get the fixes out.

Thanks again for bringing this to our attention!

Aneesh

[anderswagner]

@aneeshk-citrix

1. Yes and No. We're trying to make our current system able to be imported and then managed from terraform with this provider going forward. Previously it was handled with API calls in some ansible code (more or less manually calling the API). We are also changing the behaviour of some things, like adding the folder path (which we did not use previously when just calling the API through ansible). The field persist_user_changes is triggering the replacement, but I can see now that it does actually not import this field when we import our current resources.

2. Then perhaps I am doing the import wrong as there is currently a resource dependency between the catalog and the delivery group (we have one delivery group for each catalog afaik, and vice versa).

This is our configuration for the delivery group:

resource "citrix_delivery_group" "default" {
  name                       = var.catalog.publishedName
  minimum_functional_level   = var.catalog.functionalLevel
  delivery_group_folder_path = "omitted"
  delivery_type              = var.catalog.deliveryType

  associated_machine_catalogs = [{
    machine_catalog = citrix_machine_catalog.default.id
    machine_count   = citrix_machine_catalog.default.provisioning_scheme.number_of_total_machines
  }]

  desktops = [{
    enabled                = true
    published_name         = var.catalog.publishedName
    enable_session_roaming = true
  }]

  autoscale_settings = {
    autoscale_enabled = true
  }

  restricted_access_users = {
    allow_list = [for group in var.catalog.deliveryCatalogGroups : "omitted" ]
  }

  # We want to ignore changes to this, as they are managed elsewhere
  lifecycle {
    ignore_changes = [
      autoscale_settings.autoscale_enabled
    ]
  }
}

[aneeshk-citrix]

@anderswagner

1. persist_user_changes is an optional property and that's why it's not being imported (it is not set in state if the plan does not have it and import does not have plan). We will work on a fix for this.
2. What you have is correct. The dependency chain is not established until after an apply is done after import. In your case, the very first apply results in a replace (that fails). So the dependency is not really established and it tries to delete the machine catalog alone.

Workaround: Skip specifying persist_user_changes in your config. Let us know how that goes.

We will address two issues:

1. Fix up persist_user_changes so that it does not result in a replace after importing.
2. Error out when deleting machine catalog containing machines still a part of delivery group (not an issue in your case but this fix is needed).

[anderswagner]

@aneeshk-citrix
Both sounds really good. Let me know if you need more information from our setup. And thank you.