Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Support a generic http datasource #6223

Closed
ChipWolf opened this issue May 13, 2020 · 4 comments
Closed

Support a generic http datasource #6223

ChipWolf opened this issue May 13, 2020 · 4 comments
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started type:feature Feature (new functionality)

Comments

@ChipWolf
Copy link

ChipWolf commented May 13, 2020

What would you like Renovate to be able to do?
A generic datasource to retrieve the latest of something from an http endpoint would be useful for a variety of applications. One application of this feature I have in mind would be for Jenkins builds.

Using an endpoint such as http://ci.aksw.org/jenkins/job/LinkedGeoData/lastSuccessfulBuild/buildNumber we can retrieve the latest build number for a given Jenkins job.

Perhaps we could use some regex to select a specific piece of text on a given page.

Describe the solution you'd like

Something like this:

{
  "extends": ["config:base"],
  "regexManagers": [
    {
      "fileMatch": ["^helloworld.yaml$"],
      "matchStrings": [
        "name: BUILD:(?<currentValue>.*)\n"
      ],
      "datasourceTemplate": "generic"
    },
    {
      "fileMatch": ["^otherfile.yaml$"],
      "matchStrings": [
        "name: (?<depName>.*):(?<currentValue>.*)\n"
      ],
      "datasourceTemplate": "generic"
    }
  ],
  "packageRules": [
    {
      "datasources": ["generic"],
      "managers": ["regex"],
      "registryUrls": ["https://ci.openshift.redhat.com/jenkins/job/push_image_inspector_images/"],
      "genericRegex": "build-row\" href=\"\/jenkins\/job\/.*?\/(?<currentValue>.*?)(?s)(?:\\R.*)"
    },
    {
      "datasources": ["generic"],
      "managers": ["regex"],
      "packageNames": [
        "my-package"
      ],
      "registryUrls": ["https://ci.openshift.redhat.com/jenkins/job/push_wildfly_images/lastSuccessfulBuild/buildNumber"]
    }
  ]
}
@rarkins rarkins added type:feature Feature (new functionality) needs-requirements priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others labels May 13, 2020
@rarkins
Copy link
Collaborator

rarkins commented May 13, 2020

It could be nice to do it "properly" and allow (a) multiple values, and (b) maybe even a deep or nested mapping of any JSON response

@rarkins
Copy link
Collaborator

rarkins commented Nov 23, 2020

Idea: make this similarly configurable as regex managers:

  • template-able registry URL
  • auto parse JSON/YAML/TOML/XML
  • allow override of parsing type if not able to auto-detect correctly
  • use JSONPATH to pinpoint package metadata and release data
  • use handlebars objects to map that data back into a compatible release

@rarkins
Copy link
Collaborator

rarkins commented May 13, 2023

Although this issue came first, I'm going to prefer #15193 to continue with

@rarkins
Copy link
Collaborator

rarkins commented May 13, 2023

Duplicate of #15193

@rarkins rarkins marked this as a duplicate of #15193 May 13, 2023
@rarkins rarkins closed this as not planned Won't fix, can't repro, duplicate, stale May 13, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 13, 2023
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started type:feature Feature (new functionality)
Projects
None yet
Development

No branches or pull requests

2 participants