aws_logs: CfnAccountPolicy Returns bad request

[LucasSymons]

Describe the bug

When creating an AccountPolicy via cdk, I get back a request error.
"Invalid request provided: AWS::Logs::AccountPolicy" (RequestToken: 717f24d1-38cd-2b20-b7a1-1b04be946d9f, HandlerErrorCode: InvalidRequest)

Testing the stack with cdk synth no errors are returned.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

Expected successful creation or an obvious error about what the problem is.

Current Behavior

Deployment failed: Error: The stack named cloudwatch-data-protection-NONPROD failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "Invalid request provided: AWS::Logs::AccountPolicy" (RequestToken: 717f24d1-38cd-2b20-b7a1-1b04be946d9f, HandlerErrorCode: InvalidRequest)
    at FullCloudFormationDeployment.monitorDeployment (/usr/local/Cellar/aws-cdk/2.154.1/libexec/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:523:13)
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
    at Object.deployStack2 [as deployStack] (/usr/local/Cellar/aws-cdk/2.154.1/libexec/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:332:24)
    at /usr/local/Cellar/aws-cdk/2.154.1/libexec/lib/node_modules/aws-cdk/lib/util/work-graph.ts:105:11

The stack named cloudwatch-data-protection-NONPROD failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Resource handler returned message: "Invalid request provided: AWS::Logs::AccountPolicy" (RequestToken: 717f24d1-38cd-2b20-b7a1-1b04be946d9f, HandlerErrorCode: InvalidRequest)

Reproduction Steps

import json
from pathlib import Path as Lib_Path

import aws_cdk as cdk
from aws_cdk import aws_logs as logs
from constructs import Construct
from private_internal_lib import EnvironmentName, Utils


class CloudwatchDataProtectionStack(cdk.Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        env_name = EnvironmentName[self.node.try_get_context("env").upper()]

        _config = Utils.get_config_from_context(
            self.node, "cloudwatch-data-protection", env_name
        )

        _audit_log_group = logs.LogGroup(
            self,
            "AuditGroup",
            log_group_name="audit-findings",
            retention=logs.RetentionDays[_config["log_retention"]],
        )

        # Load Policy Doc
        dirname = Lib_Path(__file__).parent
        with Lib_Path.open(Lib_Path(dirname / "policy.json")) as json_file:
            json_data = json.load(json_file)

        _cfn_account_policy = logs.CfnAccountPolicy(
            self,
            "MyCfnAccountPolicy",
            policy_document=json.dumps(json_data),
            policy_name="DATAPROTECTION",
            policy_type="DATA_PROTECTION_POLICY",
            # the properties below are optional
            scope="ALL",
        )

The Policy file in use is below

{
    "Name": "ACCOUNT_DATA_PROTECTION_POLICY",
    "Description": "",
    "Version": "2021-06-01",
    "Statement": [
        {
            "Sid": "audit-policy",
            "DataIdentifier": [
                "arn:aws:dataprotection::aws:data-identifier/CreditCardMagneticStripe",
                "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber",
                "arn:aws:dataprotection::aws:data-identifier/CreditCardSecurityCode",
                "arn:aws:dataprotection::aws:data-identifier/DriversLicense-AU",
                "arn:aws:dataprotection::aws:data-identifier/EmailAddress",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-CA",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-DE",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-ES",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-FR",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-GB",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-IT",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-US",
                "arn:aws:dataprotection::aws:data-identifier/Address",
                "Medicare"
            ],
            "Operation": {
                "Audit": {
                    "FindingsDestination": {
                        "CloudWatchLogs": {
                            "LogGroup": "/audit-findings"
                        }
                    }
                }
            }
        },
        {
            "Sid": "redact-policy",
            "DataIdentifier": [
                "arn:aws:dataprotection::aws:data-identifier/CreditCardMagneticStripe",
                "arn:aws:dataprotection::aws:data-identifier/CreditCardNumber",
                "arn:aws:dataprotection::aws:data-identifier/CreditCardSecurityCode",
                "arn:aws:dataprotection::aws:data-identifier/DriversLicense-AU",
                "arn:aws:dataprotection::aws:data-identifier/EmailAddress",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-CA",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-DE",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-ES",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-FR",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-GB",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-IT",
                "arn:aws:dataprotection::aws:data-identifier/PassportNumber-US",
                "arn:aws:dataprotection::aws:data-identifier/Address",
                "Medicare"
            ],
            "Operation": {
                "Deidentify": {
                    "MaskConfig": {}
                }
            }
        }
    ],
    "Configuration": {
        "CustomDataIdentifier": [
            {
                "Name": "Medicare",
                "Regex": "^[2-6]{1}\\d{3}[ ]?\\d{5}[ ]?\\d{1}[- ]?\\d?"
            }
        ]
    }
}

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.154.1 (build febce9d)

Framework Version

No response

Node.js Version

v20.10.0

OS

MacOS 14.6.1

Language

Python

Language Version

No response

Other information

No response

[khushail]

Hi @LucasSymons , thanks for reporting this.
I am able to repro the issue given this code -

                with open('/policy_doc.json', 'r') as file:
                    data = json.load(file)
          
                cfn_account_policy = logs.CfnAccountPolicy(self, "MyCfnAccountPolicy",
                policy_document=json.dumps(data),
                policy_name="policyName",
                policy_type="DATA_PROTECTION_POLICY",
                scope="ALL",
                )

Although the cdk synth succeeded but error observed during cdk deploy -

Since CfnAccountPolicy is L1 construct , this is a Cloudformation issue.

[khushail]

Internal ticket filed with Cloudformation support - P151743364

[garysassano]

I experienced the same exact issue yesterday when trying to create the following resource:

new CfnAccountPolicy(this, "LogsAccountPolicy", {
  policyName: "LambdaSubscriptionPolicy",
  policyDocument: JSON.stringify({
    destinationArn: logProcessorFunction.functionArn,
    filterPattern: "{ $.__otel_otlp_stdout = * }",
    distribution: "Random",
  }),
  policyType: "SUBSCRIPTION_FILTER_POLICY",
  scope: "ALL",
  selectionCriteria: `LogGroupName NOT IN ["/aws/${logProcessorFunction.functionName}"]`,
});

The equivalent SAM template was working just fine:

LogsAccountPolicy:
  Type: AWS::Logs::AccountPolicy
  Properties:
    PolicyName: "LambdaSubscriptionPolicy"
    PolicyDocument: 
      Fn::Sub: |
        {
          "DestinationArn": "${LogProcessorFunction.Arn}",
          "FilterPattern": "{ $.__otel_otlp_stdout = * }",
          "Distribution": "Random"
        }
    PolicyType: "SUBSCRIPTION_FILTER_POLICY"
    Scope: "ALL"
    SelectionCriteria: !Sub "LogGroupName NOT IN [\"/aws/${LogProcessorFunction}\"]"