cloudnative wiki

Azure Key Vault Architecture, Managed HSM, and Cryptographic Governance

9 min read

Azure Key Vault Architecture, Managed HSM, and Cryptographic Governance 🔐🏛️

Azure Key Vault is Microsoft’s cloud-native hardware security and secrets management service for securely storing, accessing, and lifecycle-managing cryptographic keys, application secrets, and TLS certificates. Offering both multi-tenant software-backed vaults and dedicated single-tenant Azure Key Vault Managed HSM (validated to FIPS 140-2 Level 3), Key Vault provides the cryptographic root of trust for Customer-Managed Encryption Keys (CMEK) across all Azure storage, database, and compute services.

1. Architecture & Vault Types

Key Vault segregates sensitive cryptographic assets into three distinct entity types, each backed by granular lifecycle and access controls.

                           APPLICATIONS / AZURE WORKLOADS
                                          │
                                          ▼ (HTTPS / Port 443)
       ┌────────────────────────────────────────────────────────────────────────┐
       │                   AZURE KEY VAULT INGESTION GATEWAY                    │
       │  - TLS termination & Microsoft Entra ID OAuth2 token validation        │
       │  - IP firewall rules & Private Link private endpoint enforcement       │
       └───────────────────────────────────┬────────────────────────────────────┘
                                           │
             ┌─────────────────────────────┴─────────────────────────────┐
             │                                                           │
             ▼ Multi-Tenant Key Vault (Standard/Premium)                 ▼ Dedicated Single-Tenant Managed HSM
    ┌─────────────────────────────────┐                 ┌─────────────────────────────────┐
    │     STANDARD / PREMIUM VAULT    │                 │      MANAGED HSM POOL (DEDICATED│
    │                                 │                 │                                 │
    │  ┌───────────────────────────┐  │                 │  ┌───────────────────────────┐  │
    │  │ SECRETS                   │  │                 │  │ FIPS 140-2 LEVEL 3 HSM    │  │
    │  │ - API keys, DB passwords  │  │                 │  │ - Dedicated hardware      │  │
    │  │ - Up to 25 KB string data │  │                 │  │ - Isolated HSM partition  │  │
    │  └───────────────────────────┘  │                 │  │ - B-Series / M-Series     │  │
    │  ┌───────────────────────────┐  │                 │  └───────────────────────────┘  │
    │  │ KEYS (RSA / EC)           │  │                 │  ┌───────────────────────────┐  │
    │  │ - Software (Standard)     │  │                 │  │ LOCAL HSM RBAC            │  │
    │  │ - HSM-backed (Premium)    │  │                 │  │ - Zero Microsoft access   │  │
    │  └───────────────────────────┘  │                 │  │ - Security domain backup  │  │
    │  ┌───────────────────────────┐  │                 │  └───────────────────────────┘  │
    │  │ CERTIFICATES (X.509)      │  │                 │                                 │
    │  │ - Automated DigiCert sync │  │                 │                                 │
    │  │ - Auto-renewal rotation   │  │                 │                                 │
    │  └───────────────────────────┘  │                 │                                 │
    └─────────────────────────────────┘                 └─────────────────────────────────┘

Key Vault Standard/Premium vs Managed HSM

DimensionKey Vault StandardKey Vault PremiumManaged HSM
Tenant ModelMulti-tenantMulti-tenantDedicated single-tenant pool
FIPS 140 ValidationFIPS 140-2 Level 1 (Software)FIPS 140-2 Level 2 (HSM)FIPS 140-2 Level 3 (Dedicated HSM)
Asset TypesKeys, Secrets, CertificatesKeys, Secrets, CertificatesCryptographic Keys only
Throughput GuaranteeShared throttling limitsShared throttling limitsDedicated, thousands of QPS
Authorization ModelAzure RBAC or Vault PoliciesAzure RBAC or Vault PoliciesLocal Data Plane RBAC only
Security DomainManaged by MicrosoftManaged by MicrosoftCustomer-controlled offline quorum
Base Hourly Pricing$0.00 base (pay per op)$0.00 base (pay per op)~3,212/month)

2. Core Concepts: Access Policies vs Azure RBAC

Historically, Key Vault used Vault Access Policies, which granted blanket permissions across all keys or all secrets in a vault (no resource-level granularity).

The Azure RBAC Permission Model (Production Standard)

Modern architectures configure Key Vault with Azure role-based access control (Azure RBAC):

  • Enables assigning permissions at the level of an individual key, secret, or certificate.
  • Leverages built-in roles:
    • Key Vault Administrator: Full management of vault and data plane.
    • Key Vault Secrets Officer: Read, write, and delete secrets.
    • Key Vault Secrets User: Read-only access to secret contents (ideal for application Managed Identities).
    • Key Vault Crypto Service Encryption User: Permission to wrap/unwrap DEKs for CMEK storage encryption.

Soft-Delete and Purge Protection

  • Soft-Delete (Mandatory): Deleted vaults and cryptographic objects remain in a recoverable state for a retention window (default: 90 days, configurable 7-90 days).
  • Purge Protection: When enabled, neither customers nor Microsoft support can permanently destroy a soft-deleted vault or key until the retention period expires. This is an essential defense against malicious insider threats and ransomware attempting to cryptographically wipe enterprise disks.

3. Production Deployment & CLI Operations (az)

1. Deploy Key Vault with Azure RBAC and Purge Protection

az group create --name rg-secops-prod --location eastus
 
# Deploy Production Key Vault with Azure RBAC and Purge Protection
az keyvault create \
    --name kv-enterprise-core-prod \
    --resource-group rg-secops-prod \
    --location eastus \
    --sku Premium \
    --enable-rbac-authorization true \
    --enable-purge-protection true \
    --retention-days 90 \
    --public-network-access Disabled

2. Grant Granular Read Access to an Application Managed Identity

# Retrieve Application System-Assigned Managed Identity Principal ID
APP_PRINCIPAL_ID=$(az webapp identity show \
    --name app-ecommerce-core-prod \
    --resource-group rg-webapps-prod \
    --query principalId -o tsv)
 
# Retrieve Key Vault Resource ID
KV_ID=$(az keyvault show \
    --name kv-enterprise-core-prod \
    --resource-group rg-secops-prod \
    --query id -o tsv)
 
# Assign 'Key Vault Secrets User' role strictly to the Web App identity
az role assignment create \
    --assignee-object-id "${APP_PRINCIPAL_ID}" \
    --assignee-principal-type ServicePrincipal \
    --role "Key Vault Secrets User" \
    --scope "${KV_ID}"

3. Store and Retrieve an Encrypted Secret

# Set a database connection string secret with activation and expiration dates
az keyvault secret set \
    --vault-name kv-enterprise-core-prod \
    --name "DatabaseConnectionString" \
    --value "Server=psql-prod.postgres.database.azure.com;Database=prod;User=admin;Password=secret" \
    --expires "2027-01-01T00:00:00Z"
 
# Read secret value via CLI
az keyvault secret show \
    --vault-name kv-enterprise-core-prod \
    --name "DatabaseConnectionString" \
    --query value -o tsv

4. Create an HSM-Backed RSA Key with Automated Rotation Policy

# Create RSA 4096-bit HSM key
az keyvault key create \
    --vault-name kv-enterprise-core-prod \
    --name "cmek-blob-storage-key" \
    --protection hsm \
    --size 4096
 
# Configure automated key rotation policy (rotate every 180 days, notify at 30 days before)
az keyvault key rotation-policy update \
    --vault-name kv-enterprise-core-prod \
    --name "cmek-blob-storage-key" \
    --value-file - <<EOF
{
  "lifetimeActions": [
    {
      "trigger": {
        "timeAfterCreate": "P180D"
      },
      "action": {
        "type": "Rotate"
      }
    },
    {
      "trigger": {
        "timeBeforeExpiry": "P30D"
      },
      "action": {
        "type": "Notify"
      }
    }
  ],
  "attributes": {
    "expiryTime": "P365D"
  }
}
EOF

5. Deploy Private Endpoint for Key Vault

# Deploy Private Endpoint into secure VNet subnet
az network private-endpoint create \
    --name pe-keyvault-prod \
    --resource-group rg-secops-prod \
    --vnet-name vnet-spoke-prod \
    --subnet snet-private-endpoints \
    --private-connection-resource-id "${KV_ID}" \
    --group-id vault \
    --connection-name conn-pe-keyvault

4. Quotas, Performance, and Configuration Limits

Parameter / DimensionKey Vault Standard / PremiumManaged HSM
Transactions per Region4,000 requests / 10 secondsThousands of dedicated QPS
Max Secret Size25 KBN/A (Keys only)
Max Certificate Size100 KBN/A (Keys only)
Max Keys / Secrets per Vault25,000 items5,000 HSM keys
Soft-Delete Retention7 to 90 days7 to 90 days
Purge ProtectionImmutable once enabledEnabled by default
Network IsolationPrivate Link & FirewallPrivate Link only

5. Official References & Documentation

6. Realistic Pricing Scenarios

Pricing components:

  1. Key Vault Standard: 0.03 per 10,000 operations.
  2. Key Vault Premium (HSM Keys): 0.03 per 10,000 operations.
  3. Managed HSM (Dedicated Pool): ~3,212/month base fee).

Scenario A: Standard Enterprise Web Application (Key Vault Standard)

  • Inventory:
    • 1 Key Vault Standard instance.
    • Stores 50 application secrets (database credentials, API keys) and 5 TLS certificates.
    • Microservices perform 500,000 secret read operations per month (with in-memory client caching).
  • Monthly Cost Calculation:
    • Base Vault: $0.00
    • Secret Operations: (500{,}000 / 10{,}000) \times \0.03 = \mathbf{$1.50}$
    • Certificate Renewal Operations (5 certs): 5 \times \3.00 = \mathbf{$15.00}$
  • Total Monthly Cost: $16.50 / month

Scenario B: Highly Regulated Financial Payment Gateway (Managed HSM)

  • Inventory:
    • Dedicated Managed HSM Pool (FIPS 140-2 Level 3) for payment tokenization and disk CMEK.
    • 100 active HSM cryptographic keys.
    • High-frequency cryptographic operations: 10 million sign/decrypt operations per month.
  • Monthly Cost Calculation:
    • Managed HSM Pool: 3,212.00**
    • Cryptographic Operations: Included in dedicated pool cost ($0.00).
  • Total Monthly Cost: $3,212.00 / month

7. Battle-Tested Nuggets & Production Gotchas

  1. The Regional Throttling Boundary (4,000 Ops / 10s): Key Vault enforces strict throttling limits of 4,000 requests per 10-second window per vault per subscription per region. If 50 Kubernetes microservice pods restart simultaneously and each attempts to read 20 database secrets on startup without caching, Key Vault returns HTTP 429 Too Many Requests. The entire microservice fleet will crash in CrashLoopBackOff. Always implement client-side secret caching using libraries like Azure.Security.KeyVault.Secrets with exponential backoff.
  2. Purge Protection Cannot Be Disabled: Once Purge Protection is enabled on an Azure Key Vault, it can NEVER be disabled by anyone, including the Global Administrator or Microsoft Engineering. If an engineer deletes a test vault with Purge Protection enabled, that vault name remains locked and soft-deleted for the full 90-day retention period; you cannot recreate a new vault with the same name until the 90 days expire. Always append unique environment suffixes to vault names (kv-app-dev-01).
  3. Vault Access Policies vs Azure RBAC Migration Gotcha: If you change a Key Vault from “Vault Access Policy” to “Azure role-based access control (Azure RBAC)”, all existing access policies are instantly superseded and invalidated. Any application relying on legacy access policies will immediately lose access with 403 Forbidden: Caller is not authorized. Ensure you pre-assign all necessary Azure RBAC roles to applications before flipping the authorization toggle.
  4. Secrets Are Not Monitored by Default: While Key Vault stores secrets securely, it does not alert anyone when an API secret or certificate is about to expire. When a production database credential expires at midnight, services crash without warning. Integrate Key Vault with Azure Event Grid to listen to Microsoft.KeyVault.SecretNearExpiry events and trigger automated PagerDuty tickets or Slack notifications 30 days prior to expiration.
  5. Private Endpoint DNS Loopback Trap: When deploying a Private Endpoint for Key Vault, ensure your VNet resolves kv-enterprise-core-prod.vault.azure.net to the private IP via the privatelink.vaultcore.azure.net Private DNS Zone. If local DNS resolution fails or returns public IP 20.x.x.x, calls to Key Vault will be rejected by the vault’s firewall with ForbiddenByFirewall: Public network access is disabled.
  6. Managed HSM Security Domain Quorum Requirement: When provisioning an Azure Managed HSM, you must download and distribute the Security Domain among 3 to 10 security officers (using an -of- Shamir’s Secret Sharing quorum, e.g., 3 of 5 keys required). If a regional disaster occurs and you need to restore your Managed HSM pool, it is physically impossible to restore without the Security Domain quorum keys. Store these quorum keys in physical safes in separate geographic locations.

Across the wiki

Graph View

Backlinks