cleanup some smoke tests

[DaanHoogland]

Description

This PR splits the test_vm_life_cycle.py test file and cleans some of the conditional test skipping. This all in an attempt to reduce intermittent integration test failures.

The reason is that a lot of tests are not basic life cycle tests and differnt failures on different platform are mostly pre-conditional instead of genuine positives.

During this work I also noticed that we might gain by eliminating tests as it seems a lot of the same functionalities are tested at different locations. I will not investigate that further or address it in this PR.

I will look at other intermittent failures as well and try to address those as well. This may happen in this PR or in separate PRs.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (08749d8) 13.12% compared to head (251a913) 13.13%.

Additional details and impacted files

@@            Coverage Diff            @@
##               4.18    #8196   +/-   ##
=========================================
  Coverage     13.12%   13.13%           
- Complexity     9141     9142    +1     
=========================================
  Files          2720     2720           
  Lines        257726   257726           
  Branches      40177    40177           
=========================================
+ Hits          33838    33847    +9     
+ Misses       219598   219588   -10     
- Partials       4290     4291    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.58%. Comparing base (3e3a0c0) to head (bcc129c).
⚠️ Report is 276 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #8196   +/-   ##
=========================================
  Coverage     16.58%   16.58%           
  Complexity    13989    13989           
=========================================
  Files          5743     5743           
  Lines        510706   510706           
  Branches      62119    62119           
=========================================
  Hits          84689    84689           
  Misses       416543   416543           
  Partials       9474     9474           

Flag	Coverage Δ
uitests	3.90% <ø> (ø)
unittests	17.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10779

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11173)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 52255 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8196-t11173-kvm-ol8.zip
Smoke tests completed. 137 look OK, 7 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	Error	0.23	test_primary_storage.py
test_03_migration_options_storage_tags	Error	0.02	test_primary_storage.py
test_01_secure_vm_migration	Error	191.29	test_secured_vm_migration.py
test_02_unsecure_vm_migration	Error	450.63	test_secured_vm_migration.py
test_04_nonsecured_to_secured_vm_migration	Error	378.93	test_secured_vm_migration.py
ContextSuite context=TestCpuCapServiceOfferings>:setup	Error	0.00	test_service_offerings.py
test_02_list_snapshots_with_removed_data_store	Error	10.64	test_snapshots.py
test_02_list_snapshots_with_removed_data_store	Error	10.64	test_snapshots.py
test_01_deploy_vm_on_specific_host	Error	0.10	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	Error	0.15	test_vm_deployment_planner.py
test_08_migrate_vm	Error	0.07	test_vm_life_cycle.py
test_13_destroy_and_expunge_vm	Error	0.03	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	Error	0.30	test_vm_strict_host_tags.py
test_02_migrate_vm_strict_tags_failure	Error	0.30	test_vm_strict_host_tags.py
test_01_restore_vm_strict_tags_success	Error	0.30	test_vm_strict_host_tags.py
test_02_restore_vm_strict_tags_failure	Error	0.30	test_vm_strict_host_tags.py
test_01_scale_vm_strict_tags_success	Error	0.29	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	Error	0.31	test_vm_strict_host_tags.py
test_01_deploy_vm_on_specific_host_without_strict_tags	Error	0.27	test_vm_strict_host_tags.py
test_02_deploy_vm_on_any_host_without_strict_tags	Error	5.72	test_vm_strict_host_tags.py
test_03_deploy_vm_on_specific_host_with_strict_tags_success	Error	0.24	test_vm_strict_host_tags.py
test_04_deploy_vm_on_any_host_with_strict_tags_success	Error	5.70	test_vm_strict_host_tags.py
test_05_deploy_vm_on_specific_host_with_strict_tags_failure	Failure	0.28	test_vm_strict_host_tags.py

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 11160

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11165

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11534)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58592 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8196-t11534-kvm-ol8.zip
Smoke tests completed. 139 look OK, 7 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_add_primary_storage_disabled_host	Error	336.07	test_primary_storage.py
test_01_add_primary_storage_disabled_host	Error	336.09	test_primary_storage.py
test_03_migration_options_storage_tags	Error	0.02	test_primary_storage.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	Failure	203.14	test_routers_network_ops.py
test_03_secured_to_nonsecured_vm_migration	Error	114.71	test_secured_vm_migration.py
test_03_secured_to_nonsecured_vm_migration	Error	114.71	test_secured_vm_migration.py
test_04_nonsecured_to_secured_vm_migration	Error	0.01	test_secured_vm_migration.py
test_02_list_snapshots_with_removed_data_store	Error	8.51	test_snapshots.py
test_02_list_snapshots_with_removed_data_store	Error	8.52	test_snapshots.py
ContextSuite context=TestISOUsage>:setup	Error	0.00	test_usage.py
test_08_migrate_vm	Error	45.02	test_vm_life_cycle.py
test_13_destroy_and_expunge_vm	Error	0.02	test_vm_life_cycle.py
test_01_migrate_vm_strict_tags_success	Error	98.02	test_vm_strict_host_tags.py

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

will have to be redone at some time