Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

[Feature] CNF Installation (7), Analyze and prepare specs for new installation process #2168

Closed
4 tasks done
kosstennbl opened this issue Oct 21, 2024 · 2 comments
Closed
4 tasks done

[Feature] CNF Installation (7), Analyze and prepare specs for new installation process #2168

kosstennbl opened this issue Oct 21, 2024 · 2 comments
Assignees
@svteb
Labels
enhancement New feature or request

Comments

@kosstennbl
Copy link
Collaborator

kosstennbl commented Oct 21, 2024 •
edited by svteb
Loading

It is needed to be ensured that all specs would work properly with new installation process. Most of the specs for conflicting specs were resolved in (5), but some things needs to be checked and done additionally:

  • Prepare needed functions in spec_helper for new installation method.
  • Add specs for installation of multi-deployment CNFs.
  • Check if all existing specs are compatible with the new, pathless way of cleanup
  • In preparation for (8), mark specs that would be removed with change to new installation method, if needed - prepare replacements for them.
@kosstennbl kosstennbl added the enhancement New feature or request label Oct 21, 2024
@svteb svteb self-assigned this Oct 29, 2024
svteb added a commit that referenced this issue Nov 1, 2024
@svteb
Test: Verify new installation process through actions.
969bdb2 
Refs: #2168
- This commit serves to verify that the new installation process
is working by passing actions.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
@svteb svteb mentioned this issue Nov 1, 2024
Closed
20 tasks
@svteb
Copy link
Collaborator

svteb commented Nov 4, 2024 •
edited
Loading

To verify that the new installation method introduced in #2165 works on all spec tests a dummy pull request was created to run all the tests via actions (#2172) .

The result of this test run can be viewed here: https://github.com/cnti-testcatalog/testsuite/actions/runs/11627964634?pr=2172

It appears that all tests behave as expected and there is no big hidden issue with the new installation method. Below is the explanation of the tests that are failing and why this is expected:

  1. helm - resolved in Test Adaptations: CNF Installation (5.3) Disable new cluster tests and adapt service_discovery #2166 and Feat: CNF Installation (5.4) Update compatibility tests #2167
  2. immutable_configmap - to be resolved as part of installation: CNF Installation (8) Transition to new installation #2171
  3. ip_addresses - resolved in Feature: CNF Installation (5.1) Test preparation for new installation #2163
  4. latest_tag - oversight during the creation of installation: CNF Installation (7) Verify new installation process #2172 , the test is passing correctly but since no CNF is created the auxiliary new_cnf_cleanup method fails due to assert (TL;DR it is working correctly)
  5. logger - runs a helm task, which are resolved in Test Adaptations: CNF Installation (5.3) Disable new cluster tests and adapt service_discovery #2166 and Feat: CNF Installation (5.4) Update compatibility tests #2167, otherwise functional.
  6. observability - resolved in configmap: CNF Installation (5.2) Remove installation configmap #2164
  7. pod_network_corruption - litmus test
  8. privileges - oversight during the creation of installation: CNF Installation (7) Verify new installation process #2172 , the test is passing correctly but I've missed renaming the cleanup methods since they weren't caught by grep (e.g. ShellCmd.run_testsuite("sample_privileged_cnf_non_whitelisted_cleanup")) (TL;DR it is working correctly)
  9. reasonable_startup_time - resolved in configmap: CNF Installation (5.2) Remove installation configmap #2164
  10. setup - to be resolved as part of installation: CNF Installation (7) Verify new installation process #2172 , certain outputs have been changed and they don't match the spec tests now
  11. task_runner - to be resolved as part of installation: CNF Installation (6) New installation process #2165 , there seems to be a problem with parsing a non-existent config (probably got broken as part of installation: CNF Installation (6) New installation process #2165 )
  12. chaos/litmus tests - to be resolved as part of installation: CNF Installation (8) Transition to new installation #2171 , file paths that no longer exist with new installation process are being used.

svteb pushed a commit that referenced this issue Nov 11, 2024
@collivier @svteb
feat: add spec tests for new installatio method
5f134ad 
refs: #2168

Signed-off-by: svteb <slavo.valko@tietoevry.com>
svteb added a commit that referenced this issue Nov 12, 2024
@svteb
Feat: Add spec tests for new installation method and fix minor bugs
5531b22 
Refs: #2168
- New setup spec tests were introduced to verify that the installation
  method from #2161 works as intended.
- Three distinct issues were discovered and resolved:
  * Helm directories with variable depth levels could not be discovered
  after initialization (resolved through File.basename).
  * Configs that weren't named cnf-testsuite.yml could not
  be discovered after initialization (resolved by changing the name
  internally when copying the file).
  * Existence of manifest folder was not checked (resolved by adding an
  if check).
- Additionally a common point of failure was added with a failure
  message that was previously not present.
- The issue #2176 was also discovered during attempted deployment of
  sample-cnfs/sample-elk-stack which was also added in this commit and
  is to be used in future spec test.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
@svteb svteb mentioned this issue Nov 12, 2024
Merged
20 tasks
svteb added a commit that referenced this issue Nov 13, 2024
@svteb
Feat: Add spec tests for new installation method and fix minor bugs
df63e7a 
Refs: #2168
- New setup spec tests were introduced to verify that the installation
  method from #2161 works as intended.
- Three distinct issues were discovered and resolved:
  * Helm directories with variable depth levels could not be discovered
  after initialization (resolved through File.basename).
  * Configs that weren't named cnf-testsuite.yml could not
  be discovered after initialization (resolved by changing the name
  internally when copying the file).
  * Existence of manifest folder was not checked (resolved by adding an
  if check).
- Additionally a common point of failure was added with a failure
  message that was previously not present.
- The issue #2176 was also discovered during attempted deployment of
  sample-cnfs/sample-elk-stack which was also added in this commit and
  is to be used in future spec test.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
svteb added a commit that referenced this issue Nov 14, 2024
@svteb
Feat: Add spec tests for new installation method and fix minor bugs
7621efb 
Refs: #2168
- New setup spec tests were introduced to verify that the installation
  method from #2161 works as intended.
- Three distinct issues were discovered and resolved:
  * Helm directories with variable depth levels could not be discovered
  after initialization (resolved through File.basename).
  * Configs that weren't named cnf-testsuite.yml could not
  be discovered after initialization (resolved by changing the name
  internally when copying the file).
  * Existence of manifest folder was not checked (resolved by adding an
  if check).
- Additionally a common point of failure was added with a failure
  message that was previously not present.
- The issue #2176 was also discovered during attempted deployment of
  sample-cnfs/sample-elk-stack which was also added in this commit and
  is to be used in future spec test.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
collivier pushed a commit that referenced this issue Nov 27, 2024
@svteb @collivier
Feat: Add spec tests for new installation method and fix minor bugs
f06c51f 
Refs: #2168
- New setup spec tests were introduced to verify that the installation
  method from #2161 works as intended.
- Three distinct issues were discovered and resolved:
  * Helm directories with variable depth levels could not be discovered
  after initialization (resolved through File.basename).
  * Configs that weren't named cnf-testsuite.yml could not
  be discovered after initialization (resolved by changing the name
  internally when copying the file).
  * Existence of manifest folder was not checked (resolved by adding an
  if check).
- Additionally a common point of failure was added with a failure
  message that was previously not present.
- The issue #2176 was also discovered during attempted deployment of
  sample-cnfs/sample-elk-stack which was also added in this commit and
  is to be used in future spec test.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
svteb added a commit that referenced this issue Nov 28, 2024
@svteb
Feat: Add spec tests for new installation method and fix minor bugs
8c37d23 
Refs: #2168
- New setup spec tests were introduced to verify that the installation
  method from #2161 works as intended.
- Three distinct issues were discovered and resolved:
  * Helm directories with variable depth levels could not be discovered
  after initialization (resolved through File.basename).
  * Configs that weren't named cnf-testsuite.yml could not
  be discovered after initialization (resolved by changing the name
  internally when copying the file).
  * Existence of manifest folder was not checked (resolved by adding an
  if check).
- Additionally a common point of failure was added with a failure
  message that was previously not present.
- The issue #2176 was also discovered during attempted deployment of
  sample-cnfs/sample-elk-stack which was also added in this commit and
  is to be used in future spec test.

Signed-off-by: svteb <slavo.valko@tietoevry.com>
@kosstennbl
Copy link
Collaborator Author

Done in #2177

# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@svteb svteb

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kosstennbl @svteb