triggers: Trigger and TriggerFunction resource fail on subsequent deployments. maybe incorrect serialization of timeout?

[sebastianplesciuc]

Describe the bug

The Trigger custom resource in CDK causes a CloudFormation error when trying to deploy. This seems to be because we're using a String value for a Number field but the error is not clear on which fields.

CloudFormation error on the PostDeploymentHandlerTrigger7245B54D Trigger resource with CREATE_FAILED:

Received response status [FAILED] from custom resource. Message returned: Error: NUMBER_VALUE can not be converted to a String SerializationException: NUMBER_VALUE can not be converted to a String, at throwDefaultError (/var/task/index.js:42:5084), at /var/task/index.js:42:5251, at de_CommandError (/var/task/index.js:73:155469), at process.processTicksAndRejections (node:internal/process/task_queues:95:5), at async /var/task/index.js:31:11378, at async /var/task/index.js:32:9394, at async /var/task/index.js:42:15907, at async /var/task/index.js:30:508546, at async bb.execute (/var/task/index.js:74:18032), at async Runtime.XUt [as handler] (/var/task/index.js:74:18374) at P (/var/task/index.js:1:1756) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Runtime.handler (/var/task/__entrypoint__.js:1:932) (RequestId: f5c1b14b-eea4-4862-8fda-9568d5cce4df)

Looking at the template JSON file:

  "PostDeploymentHandlerTrigger7245B54D": {
   "Type": "Custom::Trigger",
   "Properties": {
    "ServiceToken": {
     "Fn::GetAtt": [
      "AWSCDKTriggerCustomResourceProviderCustomResourceProviderHandler97BECD91",
      "Arn"
     ]
    },
    "HandlerArn": {
     "Ref": "PostDeploymentHandlerCurrentVersion0ABAF863ee3b74d8952aafc71fa7460e67144692"
    },
    "InvocationType": "RequestResponse",
    "Timeout": "900000", 
    "ExecuteOnHandlerChange": true
   },
   "UpdateReplacePolicy": "Delete",
   "DeletionPolicy": "Delete",
   "Metadata": {
    "aws:cdk:path": "REDACTED"
   }
  },

When compared to other resources like:

"AWSCDKTriggerCustomResourceProviderCustomResourceProviderHandler97BECD91": {
   "Type": "AWS::Lambda::Function",
   "Properties": {
    "Code": {
     "S3Bucket": {
      "Fn::Sub": "cdk-ff31197769-assets-${AWS::AccountId}-eu-west-2"
     },
     "S3Key": "redacted.zip"
    },
    "Timeout": 900, 
    "MemorySize": 128,
    "Handler": "__entrypoint__.handler",
    "Role": {
     "Fn::GetAtt": [
      "AWSCDKTriggerCustomResourceProviderCustomResourceProviderRoleE18FAF0A",
      "Arn"
     ]
    },
    "Runtime": "nodejs20.x"
   },
   "DependsOn": [
    "AWSCDKTriggerCustomResourceProviderCustomResourceProviderRoleE18FAF0A"
   ],
   "Metadata": {
    "aws:cdk:path": "REDACTED",
    "aws:asset:path": "../REDACTED",
    "aws:asset:property": "Code"
   }
  },

Seems to be because of toString usage here:

aws-cdk/packages/aws-cdk-lib/triggers/lib/trigger.ts

Line 131 in ce7a8d5

Timeout: props.timeout?.toMilliseconds().toString() ?? Duration.minutes(2).toMilliseconds().toString(),

It's also strange because the first time the deployment ran successfully.

Expected Behavior

Template file contains number for timeout instead of string. Deployment succeeds. Function is triggered.

Current Behavior

Deployment failed with CloudFormation Error:

Received response status [FAILED] from custom resource. Message returned: Error: NUMBER_VALUE can not be converted to a String SerializationException: NUMBER_VALUE can not be converted to a String, at throwDefaultError (/var/task/index.js:42:5084), at /var/task/index.js:42:5251, at de_CommandError (/var/task/index.js:73:155469), at process.processTicksAndRejections (node:internal/process/task_queues:95:5), at async /var/task/index.js:31:11378, at async /var/task/index.js:32:9394, at async /var/task/index.js:42:15907, at async /var/task/index.js:30:508546, at async bb.execute (/var/task/index.js:74:18032), at async Runtime.XUt [as handler] (/var/task/index.js:74:18374) at P (/var/task/index.js:1:1756) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Runtime.handler (/var/task/__entrypoint__.js:1:932) (RequestId: f5c1b14b-eea4-4862-8fda-9568d5cce4df)

Reproduction Steps

import { TriggerFunction } from 'aws-cdk-lib/triggers';
import { Duration } from 'aws-cdk-lib';
import { Code, Runtime } from 'aws-cdk-lib/aws-lambda';

new TriggerFunction(this, 'testHandler', {
      functionName: 'test',
      code: Code.fromInline('console.log("here")'),
      timeout: Duration.minutes(15),
      runtime: Runtime.NODEJS_20_X,
      environment: {
        // The timestamp causes a configuration change such that the function is executed every time the stack is updated.
        APP_DEPLOYMENT_TIMESTAMP: new Date().toISOString(),
      },
      executeOnHandlerChange: true, // Explicitly execute when code or config changes for the handler
    });

Possible Solution

Potentially, if the timeout field is the issue, fix it such that it's synthesized with a number instead of a string for the timeout field.

Additional Information/Context

No response

CDK CLI Version

2.149.0 (build c8e5924)

Framework Version

2.149.0

Node.js Version

20

OS

MacOS Sonoma 14.5 (23F79)

Language

TypeScript

Language Version

5.0.4

Other information

No response

[sebastianplesciuc]

Now looking at it, the timeout thing seems to be a red herring.

It is a parameter to the Trigger Custom Resource handler and looking at the mangled code for it (because i can't find it elsewhere) parses it to an integer: n=parseInt(t); if(isNaN(n))throw new Error(The "Timeout" property with value ${t} is not parsable to a number);.

I really can't tell what's causing CloudFormation to output that error.

[sebastianplesciuc]

Founds the logs:

{
  invokeRequest: {
    FunctionName: 'arn:aws:lambda:eu-west-2:<REDACTED>:function:<REDACTED>-PostDeployment:12',
    InvocationType: 'RequestResponse'
  }
}

{
  invokeResponse: {
    '$metadata': {
      httpStatusCode: 200,
      requestId: '4cbc5040-d804-425c-bb1c-d6b950ec7b8b',
      extendedRequestId: undefined,
      cfId: undefined,
      attempts: 2,
      totalRetryDelay: 24
    },
    FunctionError: 'Unhandled',
    ExecutedVersion: '12',
    Payload: '{"errorType":"SerializationException","errorMessage":"NUMBER_VALUE can not be converted to a String","trace":["SerializationException: NUMBER_VALUE can not be converted to a String","    at throwDefaultError (/var/task/index.js:42:5084)","    at /var/task/index.js:42:5251","    at de_CommandError (/var/task/index.js:73:155469)","    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","    at async /var/task/index.js:31:11378","    at async /var/task/index.js:32:9394","    at async /var/task/index.js:42:15907","    at async /var/task/index.js:30:508546","    at async bb.execute (/var/task/index.js:74:18032)","    at async Runtime.XUt [as handler] (/var/task/index.js:74:18374)"]}',
    StatusCode: 200
  }

{
    "errorType": "SerializationException",
    "errorMessage": "NUMBER_VALUE can not be converted to a String",
    "trace": [
        "SerializationException: NUMBER_VALUE can not be converted to a String",
        "    at throwDefaultError (/var/task/index.js:42:5084)",
        "    at /var/task/index.js:42:5251",
        "    at de_CommandError (/var/task/index.js:73:155469)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
        "    at async /var/task/index.js:31:11378",
        "    at async /var/task/index.js:32:9394",
        "    at async /var/task/index.js:42:15907",
        "    at async /var/task/index.js:30:508546",
        "    at async bb.execute (/var/task/index.js:74:18032)",
        "    at async Runtime.XUt [as handler] (/var/task/index.js:74:18374)"
    ]
}

[sebastianplesciuc]

Resource re-creation also does not seem to work. But roll-back seems to work fine. Also, deletion worked.

[sebastianplesciuc]

Well, this was completely my fault. An API Call to UpdateFunctionConfigurationCommand was sent from within the handler that was failing because an Env var was number, not string.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[aws-cdk-automation]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.