API: Fix listing zones and stoage pools

[Pearl1594]

Description

This PR fixes an issue observed upstream when listing zones and storage pools due to missing column in view due to a recent change #7489

2023-12-05 20:48:10,098 ERROR [c.c.a.ApiServer] (qtp925308434-24:ctx-e45bb790 ctx-d27d94a5) (logid:8cc1db1d) unhandled exception executing api command: [Ljava.lang.String;@3de943f8
com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT storage_pool_view.id, storage_pool_view.uuid, storage_pool_view.name, storage_pool_view.path, storage_pool_view.host_address, storage_pool_view.status, storage_pool_view.pool_type, storage_pool_view.created, storage_pool_view.removed, storage_pool_view.capacity_bytes, storage_pool_view.cluster_id, storage_pool_view.cluster_uuid, storage_pool_view.cluster_name, storage_pool_view.cluster_type, storage_pool_view.data_center_id, storage_pool_view.data_center_uuid, storage_pool_view.data_center_name, storage_pool_view.pod_id, storage_pool_view.pod_uuid, storage_pool_view.pod_name, storage_pool_view.tag, storage_pool_view.is_tag_a_rule, storage_pool_view.disk_used_capacity, storage_pool_view.disk_reserved_capacity, storage_pool_view.job_id, storage_pool_view.job_uuid, storage_pool_view.job_status, storage_pool_view.scope, storage_pool_view.capacity_iops, storage_pool_view.hypervisor, storage_pool_view.storage_provider_name, storage_pool_view.parent FROM storage_pool_view WHERE storage_pool_view.id=1
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:424)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:360)
        at com.cloud.api.query.dao.StoragePoolJoinDaoImpl.searchByIds(StoragePoolJoinDaoImpl.java:306)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        ....
Caused by: java.sql.SQLSyntaxErrorException: Unknown column 'storage_pool_view.is_tag_a_rule' in 'field list'
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
        ... 70 more

com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.cj.jdbc.ClientPreparedStatement: SELECT host_view.id, host_view.uuid, host_view.name, host_view.status, host_view.type, host_view.private_ip_address, host_view.disconnected, host_view.version, host_view.hypervisor_type, host_view.hypervisor_version, host_view.capabilities, host_view.last_ping, host_view.created, host_view.removed, host_view.oobm_enabled, host_view.oobm_power_state, host_view.ha_enabled, host_view.ha_state, host_view.ha_provider, host_view.resource_state, host_view.mgmt_server_id, host_view.cpu_sockets, host_view.cpus, host_view.speed, host_view.ram, host_view.cluster_id, host_view.cluster_uuid, host_view.cluster_name, host_view.cluster_type, host_view.data_center_id, host_view.data_center_uuid, host_view.data_center_name, host_view.pod_id, host_view.pod_uuid, host_view.pod_name, host_view.guest_os_category_id, host_view.guest_os_category_uuid, host_view.guest_os_category_name, host_view.tag, host_view.is_tag_a_rule, host_view.memory_used_capacity, host_view.memory_reserved_capacity, host_view.cpu_used_capacity, host_view.cpu_reserved_capacity, host_view.job_id, host_view.job_uuid, host_view.job_status, host_view.annotation, host_view.last_annotated, host_view.username FROM host_view WHERE host_view.id IN (1,2,3,4,5)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:424)
        at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(GenericDaoBase.java:360)
        at com.cloud.api.query.dao.HostJoinDaoImpl.searchByIds(HostJoinDaoImpl.java:462)
        at jdk.internal.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)

...
Caused by: java.sql.SQLSyntaxErrorException: Unknown column 'host_view.is_tag_a_rule' in 'field list'
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
        ... 70 more

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a15b706) 22.25% compared to head (36531eb) 28.09%.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8311      +/-   ##
============================================
+ Coverage     22.25%   28.09%   +5.84%     
- Complexity    22439    29976    +7537     
============================================
  Files          5117     5251     +134     
  Lines        346819   368789   +21970     
  Branches      49790    53761    +3971     
============================================
+ Hits          77190   103620   +26430     
+ Misses       258466   250907    -7559     
- Partials      11163    14262    +3099     

Flag	Coverage Δ
simulator-marvin-tests	23.68% <ø> (-0.15%)	⬇️
uitests	4.44% <ø> (ø)
unit-tests	14.90% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

The version of Java (11.0.21) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

[shwstppr]

@blueorangutan package

[blueorangutan]

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[weizhouapache]

code lgtm

[shwstppr]

@Pearl1594 does this need to go in engine/schema/src/main/resources/META-INF/db/views/cloud.host_view.sql based on the new pattern introduced in #7417 ?

[weizhouapache]

@Pearl1594 does this need to go in engine/schema/src/main/resources/META-INF/db/views/cloud.host_view.sql based on the new pattern introduced in #7417 ?

good point @shwstppr
:+1

[DaanHoogland]

This PR fixes an issue observed upstream when listing zones and storage pools due to missing column in view due to a

upstream :D

seriously @shwstppr is right about the location of the change. Will take some getting used to for all of us, but then it will also help us in reviewing.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 7941

[Pearl1594]

Thanks @shwstppr . I'll update the PR. I wasn't aware of these changes.

[JoaoJandre]

@Pearl1594 I've just used a diff checker to compare this PR's view with what is in #7131, and I don't see the difference, could you explain what changed?

[Pearl1594]

@Pearl1594 I've just used a diff checker to compare this PR's view with what is in #7131, and I don't see the difference, could you explain what changed?

@JoaoJandre My bad, I did not realize that we had separate sql files for views. And I do notice that the column is specified there. So, I am not really sure though why it didn't get applied, because zone and storage pool tabs do not load anymore.

[JoaoJandre]

@Pearl1594 I've just used a diff checker to compare this PR's view with what is in #7131, and I don't see the difference, could you explain what changed?

@JoaoJandre My bad, I did not realize that we had separate sql files for views. And I do notice that the column is specified there. So, I am not really sure though why it didn't get applied, because zone and storage pool tabs do not load anymore.

The view scripts are only executed on upgrades, so if your environment was already on version 4.19, they would not get updated. Also, there are some logs on DatabaseUpgradeChecker.java#executeViewScripts that show what is being executed, if you don't see any logs about that, the views did not get updated.

[Pearl1594]

Ok, seems like there was an issue during setup of the env. It was a new env based off main branch - but another env got deployed fine. Closing this PR.