Feature Better configuration management

[mohityadav766]

Adding a configSource property to OpenMetadata settings that lets operators control where
configuration is read from:

• ENV - Always read from YAML/environment variables (UI shows read-only indicator, API rejects
  updates)
• DB - Always read from database (current default behavior)
• AUTO - Last-write-wins based on timestamps

Problem solved: Previously, settings stored in the database on first startup would ignore
subsequent ENV/YAML changes, confusing operators who expected config file changes to take effect.

Scope:

• Dual-source settings (auth, email, SCIM, etc.) default to AUTO mode
• DB-only settings (login config, search settings, etc.) default to DB mode
• UI shows "Managed by Environment" indicator and hides edit buttons for ENV-managed settings
• Backend validates and rejects API updates when configSource=ENV

[github-actions]

TypeScript types have been updated based on the JSON schema changes in the PR

[github-actions]

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	65.87% (55672/84517)	44.97% (28795/64030)	47.83% (8763/18323)

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ui'

Issues
9 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[gitar-bot]

⚠️ Bug: Race condition in lazy initialization of searchRepository

The getSearchRepository() method implements lazy initialization with a check-then-act pattern that has two issues:

1. Missing double-check inside synchronized block: After thread A checks searchRepository == null but before it acquires the lock, thread B might initialize it. Thread A then enters the synchronized block and creates a second instance, wasting resources and potentially causing issues with any state in SearchRepository.

2. Non-volatile static field: Even with proper double-checked locking, without volatile on the searchRepository field, there's a risk of another thread seeing a partially constructed object due to Java's memory model.

Impact: In a multi-threaded server environment (which OpenMetadata is), multiple threads calling getSearchRepository() during startup could result in multiple SearchRepository instances being created, or worse, one thread using a partially initialized instance.

Suggested fix:

• Add volatile to the searchRepository field declaration
• Add a second null check inside the synchronized block after acquiring the lock

_{Was this helpful? React with 👍 / 👎}

[gitar-bot]

🔍 CI failure analysis for ac6b5ad: Multiple CI failures: Backend integration tests failed during bootstrap (RELATED to PR). maven-postgresql-ci and maven-sonarcloud-ci both had 211 test failures (cascade). Python tests failed with Pydantic errors (RELATED). UI and Playwright had unrelated failures.

Issue

Multiple CI jobs failed across backend integration tests, Python integration tests, frontend UI coverage tests, and Playwright E2E tests.

Root Cause - Backend Integration Tests (RELATED TO PR)

During migration v1.10.2, SystemRepository.updateSetting() attempts to access OpenMetadataApplicationConfigHolder.getInstance() before initialization:

Stack trace:

java.lang.IllegalStateException: OpenMetadataApplicationConfig has not been initialized. Call initialize() first.
	at org.openmetadata.service.OpenMetadataApplicationConfigHolder.getInstance(OpenMetadataApplicationConfigHolder.java:38)
	at org.openmetadata.service.jdbi3.SystemRepository.updateSetting(SystemRepository.java:522)
	at org.openmetadata.service.jdbi3.SystemRepository.createNewSetting(SystemRepository.java:472)
	at org.openmetadata.service.migration.utils.v1102.MigrationUtil.loadDefaultSearchSettings(MigrationUtil.java:79)

Backend test suite failures (all cascade from same root cause):

• maven-postgresql-ci: 7907 tests run, 701 skipped, 211 failed
• maven-sonarcloud-ci: 7907 tests run, 701 skipped, 211 failed
• Common errors:
  • IllegalStateException: RdfRepository not initialized
  • ExceptionInInitializerError in ImpersonationTest
  • Connection refused errors for Elasticsearch
  • Search index not available in SearchResourceTest

These 211 failures in each suite are cascade effects from the same bootstrap initialization issue. Migration fails, leaving system inconsistent.

Root Cause - Python Integration Tests (RELATED TO PR)

2 Python integration tests failed with Pydantic validation errors on both Python 3.10 and 3.11:

• test_ometa_server_api.py::OMetaGlossaryTest::test_profiler_configuration
• test_sqa_profiler.py::TestSQAProfiler::test_profiler_workflow_w_globale_config

Test results: 2 failed, 531 passed, 21 skipped, 2 errors

Error:

pydantic_core._pydantic_core.ValidationError: 64 validation errors for Settings
config_value.configSource
  Extra inputs are not permitted [type=extra_forbidden, input_value='DB', input_type=str]

Assessment: Backend returns configSource field, but Python Pydantic models haven't been regenerated.

Root Cause - UI Coverage Tests (UNRELATED)

6 UI tests failed with MUI ownerState prop warnings - pre-existing MUI v7 migration issues.

Root Cause - Playwright E2E Tests (UNRELATED)

6 total failures across 2 shards - typical E2E flakiness.

Details

Impact:

• integration-tests-postgres-opensearch: Bootstrap failed, 0 tests (RELATED)
• integration-tests-mysql-elasticsearch: Bootstrap failed, 0 tests (RELATED)
• maven-postgresql-ci: 211 tests failed (RELATED)
• maven-sonarcloud-ci: 211 tests failed (RELATED)
• py-run-tests (3.10 & 3.11): 2 tests failed (RELATED)
• maven-collate-ci: Cascade failure (RELATED)
• ui-coverage-tests: 6 tests failed (UNRELATED)
• playwright-ci-postgresql: 6 tests failed (UNRELATED)
• Test Report jobs: Cascade failures (RELATED)

Code Review ⚠️ Changes requested 1 resolved / 4 findings

Good feature implementation for configuration management. One race condition found in the lazy initialization of searchRepository, plus two minor quality issues from the previous review remain unresolved (unused parameter in shouldUseEnvValue, IllegalArgumentException for config protection).

⚠️ Bug: Race condition in lazy initialization of searchRepository

📄 openmetadata-service/src/main/java/org/openmetadata/service/Entity.java:892-893

The getSearchRepository() method implements lazy initialization with a check-then-act pattern that has two issues:

1. Missing double-check inside synchronized block: After thread A checks searchRepository == null but before it acquires the lock, thread B might initialize it. Thread A then enters the synchronized block and creates a second instance, wasting resources and potentially causing issues with any state in SearchRepository.

2. Non-volatile static field: Even with proper double-checked locking, without volatile on the searchRepository field, there's a risk of another thread seeing a partially constructed object due to Java's memory model.

Impact: In a multi-threaded server environment (which OpenMetadata is), multiple threads calling getSearchRepository() during startup could result in multiple SearchRepository instances being created, or worse, one thread using a partially initialized instance.

Suggested fix:

• Add volatile to the searchRepository field declaration
• Add a second null check inside the synchronized block after acquiring the lock

💡 Quality: IllegalArgumentException may not be appropriate HTTP status for config protection

📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/system/SystemResource.java 📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/system/SystemResource.java

When a user tries to update an ENV-managed configuration, the code throws IllegalArgumentException which typically maps to HTTP 400 (Bad Request). However, this scenario might be better represented as HTTP 403 (Forbidden) or HTTP 409 (Conflict) since:

1. The request format is valid (not a "bad request")
2. The user is authorized but the operation is forbidden due to configuration policy
3. There's a conflict between the requested action and the system's configuration state

Suggested fix: Consider using a more specific exception that maps to the appropriate HTTP status code, such as:

• Custom exception mapped to 403 Forbidden, or
• Use jakarta.ws.rs.ForbiddenException if the intent is to indicate access denial
• Use a custom exception mapped to 409 Conflict if indicating a state conflict

This would provide clearer semantics to API consumers about why their request was rejected.

💡 Bug: envSyncTimestamp parameter is always null in shouldUseEnvValue call

📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/settings/SettingsCache.java

In the syncConfigWithSource method, shouldUseEnvValue is called with null for the envSyncTimestamp parameter (4th parameter), but envSyncTimestamp is never retrieved from the database even though the column exists and a getter method getEnvSyncTimestamp is defined.

boolean shouldUseEnv =
    ConfigSourceResolver.shouldUseEnvValue(
        configSource,
        currentEnvHash,
        storedEnvHash,
        null,  // envSyncTimestamp is always null
        dbModifiedTimestamp,
        applicationStartTime);

While the current implementation of shouldUseEnvValue doesn't use envSyncTimestamp (it's accepted but not referenced in the method body), passing null suggests this was either forgotten or left for future implementation. If this parameter is intended to be used, the method should retrieve it via Entity.getSystemRepository().getEnvSyncTimestamp(settingsType.toString()).

Impact: Low currently since the parameter isn't used, but this could cause issues if the logic is extended to use envSyncTimestamp in the future.

✅ 1 resolved

✅ Edge Case: Incomplete SettingsType handling in convertToOpenMetadataConfig

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/SystemRepository.java
The convertToOpenMetadataConfig method has a switch statement that only handles 8 SettingsType values (EMAIL_CONFIGURATION, AUTHENTICATION_CONFIGURATION, AUTHORIZER_CONFIGURATION, etc.) but returns null for all other cases via the default branch.

This means configurations like LINEAGE_SETTINGS, LOGIN_CONFIGURATION, OPEN_LINEAGE_SETTINGS, and ENTITY_RULES_SETTINGS (which also have configSource added in the schema changes) will bypass the ENV protection check in isUpdateAllowed() because convertToOpenMetadataConfig returns null, causing isUpdateAllowed() to return true.

Impact: Users could potentially modify settings via API even when they should be protected by configSource=ENV.

Suggested fix: Add all configuration types that implement OpenMetadataConfig to the switch statement, or use reflection/interface detection to handle all types generically.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}