CodePipelineWaitCondition - WaitCondition received failed message: 'CodePipeline Deploy ended' for uniqueId: b2507ece-e0dc-48c3-be9f-131ce2dca6fc -

[waseemshaik62913]

Hi @AWS-Joe-A,

I tried implementing this one observability workshop on our AWS account, first time it got successfully implemented and was able to access, petsite, all the things were going perfectly. Suddenly after couple of hours I retry to access petsite it was down. So i deleted stack using cli command and some time after realised it was not properly deleted. Some resources left as it was. So i deleted manually. Now again when i tried to deploy this workshop i am getting the following issue:

The workshop status into ROLLBACK_COMPLETE below are the snippets:

Checked Code pipeline, there is no pipeline running into it with the name observability workshop.

Please guide me how i can troublehsoot and resolve this issue ASAP.

Regards,
Waseem Shaik

[AWS-Joe-A]

Hello @waseemshaik62913,
You can find the Code Pipeline logs in the CloudWatch Log Group /codebuild/PipelineDeployProject
Can you review those logs and let me know what you find.

[waseemshaik62913]

@AWS-Joe-A ,

Please find the logs from cloudwatch log group [/aws/lambda/Observability-Workshop-CodePipelineReadyFunction-NoXKlRFysD7N]

Logs:

	2025-05-23T06:07:24.117Z[INFO] 2025-05-23T06:07:24.117Z 16f921fa-fd66-4a5e-b300-362f74162569 Received event{'version': '0', 'id': 'e6cf7b8f-0710-7462-6838-f7669cbf1be1', 'detail-type': 'CodePipeline Pipeline Execution State Change', 'source': 'aws.codepipeline', 'account': '724772072892', 'time': '2025-05-23T06:07:16Z', 'region': 'ap-south-1', 'resources': ['arn:aws:codepipeline:ap-south-1:724772072892:Observability-Workshop-Pipeline-H0qSN28gRrcF'], 'detail': {'pipeline': 'Observability-Workshop-Pipeline-H0qSN28gRrcF', 'execution-id': 'e5c01c71-895e-4d5b-8b49-5568c79e98d4', 'start-time': '2025-05-23T05:20:55.741Z', 'state': 'FAILED', 'version': 1.0, 'pipeline-execution-attempt': 1.0}}	[INFO] 2025-05-23T06:07:24.117Z 16f921fa-fd66-4a5e-b300-362f74162569 Received event{'version': '0', 'id': 'e6cf7b8f-0710-7462-6838-f7669cbf1be1', 'detail-type': 'CodePipeline Pipeline Execution State Change', 'source': 'aws.codepipeline', 'account': '724772072892', 'time': '2025-05-23T06:07:16Z', 'region': 'ap-south-1', 'resources': ['arn:aws:codepipeline:ap-south-1:724772072892:Observability-Workshop-Pipeline-H0qSN28gRrcF'], 'detail': {'pipeline': 'Observability-Workshop-Pipeline-H0qSN28gRrcF', 'execution-id': 'e5c01c71-895e-4d5b-8b49-5568c79e98d4', 'start-time': '2025-05-23T05:20:55.741Z', 'state': 'FAILED', 'version': 1.0, 'pipeline-execution-attempt': 1.0}}
	2025-05-23T06:07:24.117Z[INFO] 2025-05-23T06:07:24.117Z 16f921fa-fd66-4a5e-b300-362f74162569 Sending response { "Status": "FAILURE", "Reason": "CodePipeline Deploy ended", "UniqueId": "b1bee9f1-b2f4-498a-b16e-4069beebbbfa", "Data": "CodePipeline Deploy ended" }

2025-05-23T06:07:24.449ZEND RequestId: 16f921fa-fd66-4a5e-b300-362f74162569	END RequestId: 16f921fa-fd66-4a5e-b300-362f74162569
	2025-05-23T06:07:24.449ZREPORT RequestId: 16f921fa-fd66-4a5e-b300-362f74162569 Duration: 332.06 ms Billed Duration: 333 ms Memory Size: 128 MB Max Memory Used: 48 MB Init Duration: 167.81 ms

Please go through and advise for a resoltuion.

Regards,
Waseem

[Kevlw-AWS]

Hi folks, I encountered the same error. In the PipelineDeployProject logs, the first create failure happens here.

Services | 218/210 | 7:01:25 AM | CREATE_FAILED        | Custom::AWSCDK-EKS-KubernetesResource       | petsite/AwsAuth/manifest/Resource/Default (petsiteAwsAuthmanifest576F9E03) Received response status [FAILED] from custom resource. Message returned: TooManyRequestsException: Rate Exceeded.

I'm attempting a second deployment

[Kevlw-AWS]

Hi team, a second deployment also failed. When deploying the load balancer, I get

Received response status [FAILED] from custom resource. Message returned: TooManyRequestsException: Rate Exceeded. at de_TooManyRequestsExceptionRes (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:4585:21) at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-lambda/dist-cjs/index.js:3965:19) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20 at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:193:18 at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38 at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22 at async invokeUserFunction (/var/task/framework.js:1:2794) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1837) (RequestId: 186a32b5-69d4-4baf-aef7-e9c97ab5c689)