Skip to main content

ADR-010: Pluggable Message Storage Providers

Status​

Accepted

Context​

Until v0.x NimBus hardcoded Cosmos DB at every layer: runtime registration, the WebApp/CLI service code, the Bicep templates, the nb infra apply CLI, and even some user-visible API names (StoragehookReceiveCosmosAsync). Customers whose approved infrastructure does not include Cosmos DB could not adopt NimBus.

Pluggable storage therefore calls for:

  1. A true SQL-only deployment (CLI + Bicep), not just runtime persistence swap.
  2. SQL Server as the first alternate provider, full replacement of the entire ICosmosDbClient surface.
  3. Symmetric package layout: NimBus.MessageStore.CosmosDb and NimBus.MessageStore.SqlServer, both implementing provider-neutral contracts.

Decision​

Contract decomposition​

The ICosmosDbClient 60+ method surface is split into provider-neutral contracts in NimBus.MessageStore.Abstractions (four originally; IServiceHealthStore was added with the platform heartbeat):

  • IMessageTrackingStore — message records, audit, resolver state, status transitions, search
  • ISubscriptionStore — endpoint subscriptions (notifications)
  • IEndpointMetadataStore — endpoint metadata, ownership, and endpoint heartbeat state (rows, per-endpoint opt-in, the platform-wide schedule, and the atomic send claim); see heartbeat.md
  • IMetricsStore — endpoint metrics, latency, time-series
  • IServiceHealthStore — liveness of the platform's own services (the Resolver), kept out of IEndpointMetadataStore because Cosmos embeds heartbeats in the endpoint document, where a "Resolver" row would read as a phantom endpoint

A convenience INimBusMessageStore aggregate remains a published compatibility surface for provider facades and third-party or legacy integrations. First-party WebApp consumers inject only the narrow contract or contracts they use. Each provider package implements every contract; mixing providers per contract is out of scope for v1.

Provider validation at builder time​

NimBusBuilder.Build() (src/NimBus.Core/Extensions/NimBusBuilder.cs) enumerates IStorageProviderRegistration registrations and throws with a clear error message when zero or more than one provider is registered. This fires before any IHostedService starts.

Cosmos provider package​

  • Project: NimBus.MessageStore.CosmosDb (renamed from NimBus.MessageStore). Existing consumers update their package reference and registration call to the new name; types keep the NimBus.MessageStore.* namespaces so using directives do not change.
  • Centralized config-key parsing (CosmosAccountEndpoint / cosmos connection string / CosmosConnection) into AddCosmosDbMessageStore().

SQL Server provider package​

  • Project: NimBus.MessageStore.SqlServer
  • Schema management: DbUp with embedded .sql resources; AutoApply and VerifyOnly provisioning modes.
  • Physical mapping: single table per concern with EndpointId discriminator
    • composite indexes. No dynamic DDL (per-endpoint container model from ADR-008 does not translate well to SQL).
  • Idempotent status writes via MERGE on natural keys.
  • Optimistic concurrency via ROWVERSION on UnresolvedEvents.
  • IHostedService-driven schema initializer runs on startup based on ProvisioningMode.

Realtime UI updates​

The Cosmos webhook (StoragehookReceiveCosmosAsync → SignalR) was the only mechanism for live UI updates. Replaced with IMessageStateChangeNotifier fired from the Resolver write-path. The Cosmos webhook remains as a Cosmos-only adapter for backwards compatibility.

Cross-account copy operator tool​

Cosmos-specific multi-account container copy is gated behind IStorageProviderCapabilities.SupportsCrossAccountCopy. SQL deployments get a clear "use SQL backup/restore" error rather than a broken UI.

CLI + Bicep​

  • nb infra apply learns --storage-provider {cosmos|sqlserver}, --sql-mode {provision|external}, --sql-connection-string, --sql-admin-login, --sql-admin-password.
  • deploy.core.bicep gates the Cosmos module on storageProvider == 'cosmos' and conditionally provisions Azure SQL via new templates/azureSql.bicep when storageProvider == 'sqlserver' && sqlMode == 'provision'.
  • deploy.webapp.bicep makes cosmosAccountEndpoint optional, adds sqlConnectionString, validates exactly one is provided.

Security amendment (2026-07-11): secret-valued CLI options were removed because process arguments are observable. External SQL connections now use NIMBUS_SQL_CONNECTION_STRING; provisioned SQL uses NIMBUS_SQL_ADMIN_PASSWORD; bootstrap identity uses NIMBUS_IDENTITY_ADMIN_PASSWORD. The non-secret --sql-admin-login option is unchanged. The CLI passes secrets to ARM through owner-only ephemeral parameter files, and nested Bicep module parameters remain secure.

Conformance test suite​

NimBus.Testing.Conformance.MessageTrackingStoreConformanceTests is an abstract MSTest base class. Each provider's test project subclasses it and supplies a fresh store via CreateStore(). The suite covers all 9 ResolutionStatus values, idempotency, status transitions, audit history, state counts, and lifecycle.

CI runs the in-memory + Cosmos-mock suites unconditionally and the SQL Server suite against a mcr.microsoft.com/mssql/server:2022-latest service container.

Consequences​

Positive​

  • A NimBus solution can be deployed with zero Cosmos resources (SC-001).
  • Existing Cosmos-backed deployments continue to work without data migration.
  • Future providers (PostgreSQL, MySQL, Azure Table Storage) implement the same four contracts and pass the same conformance suite — no further architecture work required.
  • Operator tooling that only Cosmos can support is gated explicitly rather than silently breaking on SQL.

Negative​

  • Two paths to maintain. CI now spins up a SQL Server container.
  • Renaming the Cosmos package is a one-time breaking change for downstream consumers — they must update their package reference and registration call (AddMessageStore → AddCosmosDbMessageStore). Namespaces are unchanged so using directives keep working.
  • WebApp services and controllers depend on the narrow storage contract(s) they use; the aggregate remains available for provider implementations and legacy integrations.

See also​

  • ADR-008: Per-endpoint Cosmos containers — SQL provider does not replicate this model; uses single-table-with-discriminator instead. See docs/storage-providers.md for the SQL schema layout.