fix(terragrunt): wrong packageName resolution for GitLab, Bitbucket and Gitea datasources

[lstoeferle]

Changes

Unfortunately I introduced a bug with my last PR for the terragrunt manager, which was released in v37.253.0.

I had a look into the issue and found the problem. It seems the GitLab, Bitbucket and Gitea datasources require only the path to the repository as a value for the packageName field (see here), where the GitTag datasource requires the whole URL to the repository as value for the packageName field.

Unfortunately I'm not able to test this on a real integration, but at least the unit-tests should cover the changes.

Additionally it seems these changes also fixed a wrongly configured unit-test for terragrunt sources with fixed IP address.

Context

#27903 (comment)

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests but ran on a real repository, or
• Both unit tests + ran on a real repository

[viceice]

otherwise LGTM

[simon-wessel]

Looking forward to this fix. Terragrunt dependencies from local VCS have been broken for the last four weeks. Existing PRs have been closed as well.

[lstoeferle]

@viceice @secustor thanks for the feedback! Is there anything left blocking this PR?

[renovate-release]

🎉 This PR is included in version 37.322.2 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 37.322.2

Your semantic-release bot 📦🚀

[mstoetzer-jambit]

@lstoeferle I'm not sure, but is this handling self-issued tls certificates correctly? Can you confirm, this is working?

[lstoeferle]

@lstoeferle I'm not sure, but is this handling self-issued tls certificates correctly? Can you confirm, this is working?

Hi @mstoetzer-jambit,
sorry, but I can't give an answer on your question, since we don't use self-issued tls certificates.

[mstoetzer-jambit]

@lstoeferle

Unfortunately we had some errors:

DEBUG: Failed to look up gitlab-tags package some-namespace/terraform-modules/aws-rds.git (repository=username/wip, packageFile=dev/eu-west-1/dev/common/rds1/terragrunt.hcl, dependency=some-namespace/terraform-modules/aws-rds.git)
DEBUG: GET https://gitlab.tld:2222/api/v4/projects/some-namespace%2Fterraform-modules%2Faws-lb.git/repository/tags?per_page=100 = (code=EPROTO, statusCode=-1 retryCount=0, duration=72) (repository=username/wip)
DEBUG: Gitlab API error (repository=username/wip)
       "err": {
         "name": "RequestError",
         "code": "EPROTO",
         "timings": {
           "start": 1714143933785,
           "socket": 1714143933785,
           "lookup": 1714143933799,
           "connect": 1714143933800,
           "upload": 1714143933857,
           "error": 1714143933857,
           "phases": {"wait": 0, "dns": 14, "tcp": 1, "request": 57, "total": 72}
         },
         "message": "write EPROTO 00D89BE60C7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:355:\n",
         "stack": "RequestError: write EPROTO 00D89BE60C7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:355:\n\n    at ClientRequest.<anonymous> (/usr/local/renovate/node_modules/.pnpm/got@11.8.6/node_modules/got/dist/source/core/index.js:970:111)\n    at Object.onceWrapper (node:events:632:26)\n    at ClientRequest.emit (node:events:529:35)\n    at ClientRequest.emit (node:domain:489:12)\n    at ClientRequest.origin.emit (/usr/local/renovate/node_modules/.pnpm/@szmarczak+http-timer@4.0.6/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)\n    at TLSSocket.socketErrorListener (node:_http_client:501:9)\n    at TLSSocket.emit (node:events:517:28)\n    at TLSSocket.emit (node:domain:489:12)\n    at emitErrorNT (node:internal/streams/destroy:151:8)\n    at emitErrorCloseNT (node:internal/streams/destroy:116:3)\n    at processTicksAndRejections (node:internal/process/task_queues:82:21)\n    at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16)",
         "options": {
           "headers": {
             "user-agent": "RenovateBot/37.325.0 (https://github.com/renovatebot/renovate)",
             "accept": "application/json",
             "authorization": "***********",
             "accept-encoding": "gzip, deflate, br"
           },
           "url": "https://gitlab.tld:2222/api/v4/projects/some-namespace%2Fterraform-modules%2Faws-lb.git/repository/tags?per_page=100",
           "hostType": "gitlab-tags",
           "username": "",
           "password": "",
           "method": "GET",
           "http2": false
         }
       }

[lstoeferle]

Hi @mstoetzer-jambit,
seems like .git in the URL leads to the error. Can you remove the .git and open the URL in your browser to double check?

@viceice may I open another PR to remove the .git ending?

[viceice]

Hi @mstoetzer-jambit,
seems like .git in the URL leads to the error. Can you remove the .git and open the URL in your browser to double check?

@viceice may I open another PR to remove the .git ending?

sure

[mstoetzer-jambit]

We are using SSH URI in terragrunt.hcl:

terraform {
  source = "git::ssh://git@gitlab.tld:2222/some-namespace/terraform-modules/aws-lb.git//alb?ref=1.1.0"
}

[lstoeferle]

As a workaround you could try without .git. Is your GitLab API served over https via the default port 443?

terraform {
-   source = "git::ssh://git@gitlab.tld:2222/some-namespace/terraform-modules/aws-lb.git//alb?ref=1.1.0"
+   source = "git::ssh://git@gitlab.tld:2222/some-namespace/terraform-modules/aws-lb//alb?ref=1.1.0"
}

[mstoetzer-jambit]

removed .git from the URL as you suggested:

Renovate failed to look up the following dependencies: Failed to look up gitlab-tags package some-namespace/terraform-modules/k8s

With Renovate Version docker.io/renovate/renovate:37.240.1, we used a configuration preset with extends in the renovate.json:

{
   "packageRules": [
      {
        "matchPackagePatterns": ["terraform-modules/k8s"],
        "matchDatasources": ["git-tags"],
        "sourceUrl": "https://gitlab.tld/some-namespace/terraform-modules/k8s"
      }
   ]
}

With that, we were able to fetch release notes correctly and dependency lookup was working

[lstoeferle]

Yes, the fix is provided here #28726. The .git part should never occur in the packageName, since it's used to resolve the GitLab API URLs.