Added deleteSQLCLI to be called before the sql-cli is cloned in case …

[MitchellGale]

Description

cloneSqlCli now depends on new task deleteSQLCLI which deletes doctest/sql-cli before it is used.

When execution of doctests is halted during execution the doctest/sql-cli directory is not deleted.

Steps to reproduce the behavior:
Run OpenSearch SQL Plugin build and stop build during doctest execution.
Run OpenSearch SQL Plugin build again and observe Execution failed for task ':doctest:cloneSqlCli'. error.

What is the expected behavior?
doctest/sql-cli should be deleted before clone is attempted.

Issues Resolved

#1675

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[Yury-Fridlyand]

Maybe don't delete and don't clone, but use pip install?

[MitchellGale]

@Yury-Fridlyand, interesting. Would that not cause issues because we are not working in venv or some other virtual environment? Differences in versions, needing to update the version etc might cause issues?

[MitchellGale]

The existing code also calls for the directory to be deleted after doctests is finished

task doctest(type: Exec, dependsOn: ['bootstrap']) {

    commandLine "$projectDir/bin/test-docs"

    doLast {
        // remove the cloned sql-cli folder
        file("$projectDir/sql-cli").deleteDir()
        println("Doctest Done")
    }
}

But I wouldn't complain moving everything to a better system.

[Yury-Fridlyand]

I think you can run pip install --upgrade opensearchsql. It will do nothing, if CLI tool is already installed.
BTW, here is good place to move doctests to another CI workflow, ref opensearch-project#915.

[MitchellGale]

Thank you! I'll switch this to a draft and implement the fix with pip install upgrade.

You're saying I should implement the split in this PR as well? I can certainly add that as well.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (integ-fix_SQLCLIDocTestDelete@9824b1c). Click here to learn what that means.
The diff coverage is n/a.

@@                       Coverage Diff                        @@
##             integ-fix_SQLCLIDocTestDelete     #271   +/-   ##
================================================================
  Coverage                                 ?   97.23%           
  Complexity                               ?     4270           
================================================================
  Files                                    ?      372           
  Lines                                    ?    10652           
  Branches                                 ?      770           
================================================================
  Hits                                     ?    10358           
  Misses                                   ?      287           
  Partials                                 ?        7           

Flag	Coverage Δ
sql-engine	97.23% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[MitchellGale]

@Yury-Fridlyand, looks like there might be some issues doing this. It seems to be possible with the commandLine command, but pip commands can be different with different operating systems. We can also call a python script to do the install, but then similar case with python script syntax being different for different OS. A fix would be to do a branch if windows, if not etc. There is a shell script already in doctest/bootstrap.sh which seems to install the sql-cli

$DIR/.venv/bin/pip install -e ./sql-cli

Perhaps this shell script venv install could be used?

[Yury-Fridlyand]

Yes, I hope so.
In order to simplify testing, you can create a separate branch and remove all workflows, but doctest. Keep it running on all platforms to ensure that it works fine.

[Yury-Fridlyand]

To track: opensearch-project#1281