commit_message causes an incorrect environment variable

[MisterBianco]

While using the cdk to create a codepipeline that has a github source action I would like to pass the environment variables down to subsequent steps.

self.source_variables = {
            "AUTHOR_DATE": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.author_date),
            "BRANCH_NAME": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.branch_name),
            "COMMIT_ID": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.commit_id),
            # "COMMIT_MESSAGE": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.commit_message),
            "COMMITTER_DATE": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.committer_date),
            "COMMIT_URL": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.commit_url),
            "REPOSITORY_NAME": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.repository_name),
        }

However, if the line for commit_message is not commented out then I am met with an error that says the format is wrong: should be [{name, type, value}]

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format: [{"name": "string", "type": "string", "value": "string"}]

the issue is not present with any other value.

Reproduction Steps

Create a simple codepipeline with a GithubSourceAction and a codebuild action like:

self.build_init_action = actions.CodeBuildAction(
            action_name="initialize",
            project=build_project,
            input=self.source,
            outputs=[],
            environment_variables=self.source_variables
        )

The deploy command will succeed but the details for the second stage will not work with the commit_message.

Error Log

No Errors except when in the console.

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format: [{"name": "string", "type": "string", "value": "string"}]

Environment

• CLI Version : 1.45
• Framework Version: 1.45
• Node.js Version: v14.3.0
• OS : manjaro linux and mac osx
• Language (Version): python 3.8.3

---

This is 🐛 Bug Report

It is equally as likely that I am just missing something wildly dumb.

I can provide cf templates at request.

[skinny85]

Hey @MisterBianco ,

thanks for opening the issue. When trying your example, I get the following template when I run cdk synth:

                EnvironmentVariables: '[{"name":"AUTHOR_DATE","type":"PLAINTEXT","value":"#{Source_GitHub_NS.AuthorDate}"},{"name":"BRANCH_NAME","type":"PLAINTEXT","value":"#{Source_GitHub_NS.BranchName}"},{"name":"COMMIT_ID","type":"PLAINTEXT","value":"#{Source_GitHub_NS.CommitId}"},{"name":"COMMIT_MESSAGE","type":"PLAINTEXT","value":"#{Source_GitHub_NS.CommitMessage}"},{"name":"COMMITTER_DATE","type":"PLAINTEXT","value":"#{Source_GitHub_NS.CommitterDate}"},{"name":"COMMIT_URL","type":"PLAINTEXT","value":"#{Source_GitHub_NS.CommitUrl}"},{"name":"REPOSITORY_NAME","type":"PLAINTEXT","value":"#{Source_GitHub_NS.RepositoryName}"}]'

That seems correct to me... I don't see why COMMIT_MESSAGE would be any different than the other values here.

You said:

The deploy command will succeed but the details for the second stage will not work with the commit_message.

Can you elaborate on that? Do you mean that cdk deploy succeeds with "COMMIT_MESSAGE": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.commit_message), but the CodePipeline fails at runtime? Or something else? If it is the CodePipeline, can you show me a screenshot of the error you get?

Thanks,
Adam

[MisterBianco]

My cdk.out shows

"EnvironmentVariables": "[{\"name\":\"AUTHOR_DATE\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.AuthorDate}\"},{\"name\":\"BRANCH_NAME\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.BranchName}\"},{\"name\":\"COMMIT_ID\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.CommitId}\"},{\"name\":\"COMMIT_MESSAGE\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.CommitMessage}\"},{\"name\":\"COMMITTER_DATE\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.CommitterDate}\"},{\"name\":\"COMMIT_URL\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.CommitUrl}\"},{\"name\":\"REPOSITORY_NAME\",\"type\":\"PLAINTEXT\",\"value\":\"#{SourceVariables.RepositoryName}\"}]"

The deploy functions perfectly, greens across the terminal. The only issue is when I go to CodePipeline and try to click on the details of the build action and it wont enter the build details. The following image is what I get.

[skinny85]

Can you show me the commit message that goes out of the source action and into the build action, that causes this?

[MisterBianco]

I would love to, what do I need to do to retrieve that information. I am very new to codepipelines... And the cdk

[skinny85]

Just look at the source action that feeds the build action - there should be the commit message there (and a direct link to the commit).

[MisterBianco]

The commit message is:

 - Bosun config files

- java onbuilds
- java full vs slim/alpine builds

- Basically everything else

- reverted a setting that forces dockerfile linting on commit
  - reasons: it was inhibiting my work. can be readded later.
- removed the version label to decrease effort in maintaining version parity.
  - I dont want to force people to update two places... they will forget.
- Everything to do with dockle
- Node image
  - No one will use it, java fanbois everywhere
- Nim image
  - No one will use it, java fanbois everywhere
- Elixir image
  - No one will use it, java fanbois everywhere
- Rust image
  - No one will use it, java fanbois everywhere

Changes to be committed:
	modified:   .editorconfig
	modified:   CHANGELOG.md
	modified:   README.md
	modified:   TODO.md
	modified:   Taskfile.yml
	modified:   alpine/Dockerfile
	deleted:    alpine/README.md
	deleted:    go/README.md
	modified:   go/alpine/Dockerfile
	new file:   go/alpine/onbuild.Dockerfile
	modified:   go/bosun.yml
	modified:   go/full/Dockerfile
	new file:   go/full/onbuild.Dockerfile
	modified:   java/bosun.yml
	modified:   java/build/gradle/Dockerfile
	new file:   java/build/gradle/onbuild.Dockerfile
	modified:   java/build/maven/Dockerfile
	new file:   java/build/maven/onbuild.Dockerfile
	modified:   java/jre/Dockerfile
	modified:   python/build/Dockerfile
	modified:   python/python/Dockerfile

[skinny85]

I wonder whether there's some character(s) in the message that make CodePipeline choke on it.

Can you try a simple experiment for me? Can you try committing and pushing an empty commit with a very simple error (EDIT: obviously meant commit) message that for sure doesn't contain any special characters? Something like:

$ git commit --allow-empty -m "Test CodePipeline empty commit"

and see if you get the same error in the CodeBuild Action? (Make sure the deployed version of the CodePipeline is the one that has this error, so with the line "COMMIT_MESSAGE": codebuild.BuildEnvironmentVariable(value=self.source_action.variables.commit_message) present).

Thanks,
Adam

[MisterBianco]

It succeeded.

[skinny85]

OK. So this is a CodePipeline issue then. I'll get in contact with the team.

[skinny85]

Hey @MisterBianco ,

I've talked to the CodePipeline team, and unfortunately I don't have good news 😕 . This is a known issue - apparently the escaping for CodePipeline Variables is incorrect for some action configurations, and CodeBuild environment variables are one of those. So basically, right now, the best advice I can give is to just not use the commitMessage variable with conjunction with CodeBuild environment variables.

If you pay for AWS support, you can open an issue about this, but I can't guarantee when it will be fixed from the CodePipeline side.

Sorry I don't have better news for you. I'm resolving this one, please let us know if you need anything else from our side for this problem.

Thanks,
Adam

[jamiepmullan]

Hey @skinny85 ! Are you aware of any fix that went in for this (before I try now)?

[skinny85]

Hey @MisterBianco - I am not aware of any fix. I can ask if you want me to...?

[MisterBianco]

A fix for this would be great. It would really help with a current project I am working on. 👍🏼

[wjax]

This is still a problem today.
Passing a commit message to a codebuild action is more than frequent.

[matt-bamberger]

I get this error when the commit message has a line break in it. I reworded the commit to remove the line break and then the code pipeline succeeded.

[jonasws]

This is still a problem. Is a possible solution for the CDK to only the first line, as to not break the pipeline due to this?

UPDATE: This is obviously not something that is the CDKs default, but any change we can get some updates into whether this is somewhere in the CodeBuild team's backlog, and will be addressed? It could be as simple as only using the first line in the event of a multi-line commit message, or truncating it in some way?

[lofcodes]

This issue still exists today when using CloudFormation and the most recent commit contains a slash (present in branch name)

An extract of the configuration:

NOT WORKING

Configuration:
  ProjectName: !Ref DeployApplicationToQAProject
  EnvironmentVariables: >-
    [
      {
        "name": "BRANCH_NAME",
        "value": "#{SourceVariables.BranchName}",
        "type": "PLAINTEXT"
      },
      {
        "name": "COMMIT_ID",
        "value": "#{SourceVariables.CommitId}",
        "type": "PLAINTEXT"
      },
      {
        "name": "COMMIT_MESSAGE",
        "value": "#{SourceVariables.CommitMessage}",
        "type": "PLAINTEXT"
      }
    ]
RunOrder: 2

WORKING

Configuration:
  ProjectName: !Ref DeployApplicationToQAProject
  EnvironmentVariables: >-
    [
      {
        "name": "BRANCH_NAME",
        "value": "#{SourceVariables.BranchName}",
        "type": "PLAINTEXT"
      },
      {
        "name": "COMMIT_ID",
        "value": "#{SourceVariables.CommitId}",
        "type": "PLAINTEXT"
      }
    ]
RunOrder: 2

[prashantb1980]

Has there been any progress on this issue? Even I have a need to utilize the CommitMessage from my Codestarconnection. However, the pipeline fails when I try to pass the CommitMessage in subsequent CodeBuild step.

[mrsan22]

Workaround mentioned in this article:

• COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions --pipeline-name <Pipeline_Name> --max-items 1 | jq -r '.pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')

https://repost.aws/questions/QU2jfrtUlyQqyQ5YuUSgIEow/unexpected-codepipline-variable-resolution-error-bug

[saurabh-git-dev]

Workaround mentioned in this article:

• COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions --pipeline-name <Pipeline_Name> --max-items 1 | jq -r '.pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')

https://repost.aws/questions/QU2jfrtUlyQqyQ5YuUSgIEow/unexpected-codepipline-variable-resolution-error-bug

Hello, Thank you for sharing this I have issue when I merge any PR. It's because I think there is special character(#) in the commit.

But I am not sure how to use this.
I am using aws-cdk. Do I need to replace "#{SourceVariables.CommitMessage}" with the provided code.