Add Azure DevOps Server support #754

vinokurig · 2025-01-20T14:28:14Z

What does this PR do?

Add Azure DevOps Server support to use with PAT ONLY, as the azure Devops Api does not support oauth2, see: Support for Microsoft Azure DevOps Server (TFS) che#23306 (comment)
Change the gitconfig storage configmap to a secret because we are going to mount sensitive data to the devworkspace system scope gitconfig, see: Change the git-userdata configmap to secret che-dashboard#1313

Depends on eclipse-che/che-dashboard#1313

Screenshot/screencast of this PR

What issues does this PR fix or reference?

fixes eclipse-che/che#23306

How to test this PR?

Deploy che with the pull request image: quay.io/eclipse/che-server:pr-754 and quay.io/eclipse/che-dashboard:pr-1313
In the Dev azure Server instance create a personal access token with full access.
Create a personal access token, use the Organization input to enter the Collection name.
Start a workspace from an azure devops server repository with a devfile.

See: workspace starts with the devfile resolve.

PR Checklist

As the author of this Pull Request I made sure that:

Release Notes

Reviewers

Reviewers, please comment how you tested the PR when approving it.

openshift-ci · 2025-01-20T14:28:37Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vinokurig

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

vinokurig · 2025-01-21T15:10:25Z

/retest

vinokurig · 2025-01-22T07:37:09Z

/retest

tolusha · 2025-01-23T11:24:08Z

...ure-devops/src/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsUrl.java

  }

  protected AzureDevOpsUrl withDevfileFilenames(List<String> devfileFilenames) {
    this.devfileFilenames.addAll(devfileFilenames);
    return this;
  }

+  public AzureDevOpsUrl withServerUrl(String serverUrl) {
+    this.serverUrl = serverUrl;


When use set serverUrl, then hostname is null, which is used later in getRepositoryLocation

The hostName does not rely on serverUrl, we set it independently:

che-server/wsmaster/che-core-api-factory-azure-devops/src/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsURLParser.java

Lines 198 to 200 in 4c865ce

return new AzureDevOpsUrl()

.withHostName(

url.startsWith("git@ssh.") ? azureDevOpsScmApiEndpointHost : URI.create(url).getHost())

tolusha · 2025-01-23T11:34:31Z

...a/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsPersonalAccessTokenFetcher.java

@@ -172,8 +172,19 @@ public Optional<Boolean> isValid(PersonalAccessToken personalAccessToken) {
  public Optional<Pair<Boolean, String>> isValid(PersonalAccessTokenParams params)
      throws ScmCommunicationException {
    if (!isValidScmServerUrl(params.getScmProviderUrl())) {
-      LOG.debug("not a valid url {} for current fetcher ", params.getScmProviderUrl());
-      return Optional.empty();
+      if (OAUTH_PROVIDER_NAME.equals(params.getScmProviderName())) {


Can we refactor this fuction?
For instance, move if (OAUTH_PROVIDER_NAME.equals(params.getScmProviderName())) { before if (!isValidScmServerUrl(params.getScmProviderUrl())) {

We need to check if the token belongs to Azure DevOps SAAS or Server first. The isValidScmServerUrl function name is a bit misleading, so I renamed it.

tolusha · 2025-01-23T11:45:18Z

...rc/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsUserDataFetcher.java

-        azureDevOpsApiClient.getUserWithPAT(
-            personalAccessToken.getToken(), personalAccessToken.getScmOrganization());
-    return new GitUserData(user.getDisplayName(), user.getEmailAddress());
+    if (personalAccessToken.getScmProviderUrl().equals("https://dev.azure.com")) {


We have AzureDevOps class for contants

tolusha · 2025-01-27T09:52:45Z

...-common/src/main/java/org/eclipse/che/api/factory/server/gitlab/AbstractGitlabUrlParser.java

@@ -112,7 +112,8 @@ private boolean isApiRequestRelevant(String repositoryUrl) {
        // belongs to Gitlab.
        gitlabApiClient.getOAuthTokenInfo("");
      } catch (ScmUnauthorizedException e) {
-        return true;
+        // the error message is a JSON if it is a response from Gitlab.
+        return e.getMessage().startsWith("{");


Do we really need this changes in context of azure?

Yes, we need to specify the unauthorized exception as Azure also returns an unauthorized exception in this case.

what is the response from Azure in this case?

...vops/src/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsURLParser.java

+
+  private Optional<Matcher> getPatternMatcherByUrl(String url) {
+    String host = URI.create(url).getHost();
+    Matcher matcher = compile(format(azureDevOpsPatternTemplate, host)).matcher(url);


...vops/src/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsURLParser.java

+    if (matcher.matches()) {
+      return Optional.of(matcher);
+    } else {
+      matcher = compile(format(azureSSHDevOpsPatternTemplate, host)).matcher(url);


...vops/src/main/java/org/eclipse/che/api/factory/server/azure/devops/AzureDevOpsURLParser.java

+      if (matcher.matches()) {
+        return Optional.of(matcher);
+      } else {
+        matcher = compile(format(azureSSHDevOpsServerPatternTemplate, host)).matcher(url);


...e/infrastructure/kubernetes/namespace/configurator/GitconfigAutomauntSecretConfigurator.java

ibuziuk

Based on today's discussion it was decided to keep the initial implementation for supporting Azure DevOps TFS Server as simple as possible:

do not make any changes on the UD and close Change the git-userdata configmap to secret che-dashboard#1313
user will need to create PAT manually
user will need to configure gitconfig manually
documentation for both Azure DevOps TFS Server and gitconfig should be updated

vinokurig · 2025-02-19T14:45:05Z

/retest

openshift-ci · 2025-02-19T16:01:27Z

@vinokurig: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/v14-gitlab-with-oauth-setup-flow	`0372ac8`	link	true	`/test v14-gitlab-with-oauth-setup-flow`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

vinokurig requested review from SDawley, ibuziuk and tolusha as code owners January 20, 2025 14:28

vinokurig marked this pull request as draft January 20, 2025 14:40

openshift-ci bot added the do-not-merge/work-in-progress label Jan 20, 2025

vinokurig force-pushed the che-23306 branch 2 times, most recently from 259113e to 4c865ce Compare January 21, 2025 12:58

vinokurig marked this pull request as ready for review January 21, 2025 13:35

openshift-ci bot removed the do-not-merge/work-in-progress label Jan 21, 2025

tolusha reviewed Jan 23, 2025

View reviewed changes

tolusha reviewed Jan 27, 2025

View reviewed changes

github-advanced-security bot found potential problems Jan 29, 2025

View reviewed changes

vinokurig closed this Feb 5, 2025

vinokurig deleted the che-23306 branch February 5, 2025 14:01

vinokurig restored the che-23306 branch February 5, 2025 14:01

vinokurig reopened this Feb 5, 2025

vinokurig force-pushed the che-23306 branch 2 times, most recently from 80c8b96 to 12721bc Compare February 6, 2025 12:00

vinokurig marked this pull request as draft February 7, 2025 08:43

openshift-ci bot added the do-not-merge/work-in-progress label Feb 7, 2025

github-advanced-security bot found potential problems Feb 8, 2025

View reviewed changes

...e/infrastructure/kubernetes/namespace/configurator/GitconfigAutomauntSecretConfigurator.java Fixed Show fixed Hide fixed

...e/infrastructure/kubernetes/namespace/configurator/GitconfigAutomauntSecretConfigurator.java Fixed Show fixed Hide fixed

vinokurig force-pushed the che-23306 branch 4 times, most recently from cfaf300 to 95d556a Compare February 10, 2025 08:40

github-advanced-security bot found potential problems Feb 10, 2025

View reviewed changes

...e/infrastructure/kubernetes/namespace/configurator/GitconfigAutomauntSecretConfigurator.java Fixed Show fixed Hide fixed

...e/infrastructure/kubernetes/namespace/configurator/GitconfigAutomauntSecretConfigurator.java Fixed Show fixed Hide fixed

vinokurig force-pushed the che-23306 branch from 95d556a to 2e1f709 Compare February 10, 2025 09:19

vinokurig force-pushed the che-23306 branch from 2e1f709 to 3531467 Compare February 10, 2025 10:53

vinokurig marked this pull request as ready for review February 10, 2025 13:06

openshift-ci bot removed the do-not-merge/work-in-progress label Feb 10, 2025

vinokurig mentioned this pull request Feb 10, 2025

Change the git-userdata configmap to secret eclipse-che/che-dashboard#1313

Closed

vinokurig force-pushed the che-23306 branch from d7be878 to bfebd1f Compare February 11, 2025 09:51

tolusha mentioned this pull request Feb 12, 2025

Cannot create personal access token under user preferences eclipse-che/che#23129

Open

vinokurig force-pushed the che-23306 branch from bfebd1f to d5941b1 Compare February 12, 2025 09:25

ibuziuk requested changes Feb 12, 2025

View reviewed changes

openshift-ci bot assigned ibuziuk Feb 12, 2025

Add Azure DevOps Server support

0372ac8

vinokurig force-pushed the che-23306 branch from fcca718 to 0372ac8 Compare February 19, 2025 12:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure DevOps Server support #754

Add Azure DevOps Server support #754

vinokurig commented Jan 20, 2025 •

edited

Loading

openshift-ci bot commented Jan 20, 2025

vinokurig commented Jan 21, 2025

vinokurig commented Jan 22, 2025

tolusha Jan 23, 2025

vinokurig Jan 24, 2025

tolusha Jan 23, 2025

vinokurig Jan 24, 2025

tolusha Jan 23, 2025

vinokurig Jan 24, 2025

tolusha Jan 27, 2025

vinokurig Jan 27, 2025

ibuziuk Feb 20, 2025

ibuziuk left a comment

vinokurig commented Feb 19, 2025

openshift-ci bot commented Feb 19, 2025 •

edited

Loading

	return new AzureDevOpsUrl()
	.withHostName(
	url.startsWith("git@ssh.") ? azureDevOpsScmApiEndpointHost : URI.create(url).getHost())

Add Azure DevOps Server support #754

Are you sure you want to change the base?

Add Azure DevOps Server support #754

Conversation

vinokurig commented Jan 20, 2025 • edited Loading

What does this PR do?

Screenshot/screencast of this PR

What issues does this PR fix or reference?

How to test this PR?

PR Checklist

Release Notes

Reviewers

openshift-ci bot commented Jan 20, 2025

vinokurig commented Jan 21, 2025

vinokurig commented Jan 22, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibuziuk left a comment

Choose a reason for hiding this comment

vinokurig commented Feb 19, 2025

openshift-ci bot commented Feb 19, 2025 • edited Loading

vinokurig commented Jan 20, 2025 •

edited

Loading

openshift-ci bot commented Feb 19, 2025 •

edited

Loading