cloudnative wiki

Crossplane for Platform APIs

9 min read

Crossplane for Platform APIs

Crossplane lets a platform team define its own Kubernetes APIs and implement them as compositions of real infrastructure. A developer creates a small Database or WebService object; Crossplane expands it into the cloud resources, IAM, networking and Kubernetes objects behind it and keeps them reconciled.

That makes it one of the main engines for the self-service layer of an internal developer platform. This note is about the platform-design side. Installation, providers and AWS authentication are covered in Crossplane on EKS.

Two audiences, two layers

developer writes                     platform team writes                    Crossplane manages
┌──────────────────┐       ┌──────────────────────────────────┐     ┌──────────────────────────┐
│ kind: Database   │       │ XRD: the schema of "Database"    │     │ RDS instance             │
│ spec:            │ ───►  │ Composition: how to build one    │ ──► │ subnet group, SG         │
│   size: small    │       │   (a pipeline of functions)      │     │ KMS key, parameter group │
│   engine: 16     │       │                                  │     │ Secret with credentials  │
└──────────────────┘       └──────────────────────────────────┘     └──────────────────────────┘
   the platform API               the implementation                    managed resources

The XRD (CompositeResourceDefinition) is the contract with developers. The Composition is the implementation and can change without the contract changing — which is the property that solves the “day-two” problem described in golden paths: tighten a default in one Composition and every existing instance is reconciled to it.

Designing the API

The hard part of Crossplane is not YAML. It is deciding what the developer-facing API should look like.

Ask for intent, not implementation.

LeakyIntent-based
instanceClass: db.r7g.largesize: medium
multiAZ: true, backupRetentionPeriod: 14tier: production
subnetIds: […], vpcSecurityGroupIds: […]Nothing — derived from the environment
storageEncrypted: true, kmsKeyId: …Nothing — always on, not negotiable

Guidelines:

  • Expose what legitimately varies between uses: size, engine version, perhaps a tier. Decide everything else.
  • Never expose a security or compliance setting as optional. If encryption is required, it is not a field.
  • Use enums with meaningful names. small | medium | large can be remapped to new instance generations later; a raw instance type cannot.
  • Report useful status: an endpoint, the name of the Secret holding credentials, a ready condition.
  • Keep it cloud-neutral only if you will really run on several clouds. Otherwise the abstraction costs more than it returns.
  • Fewer fields is better. Every field is a promise you must keep across versions.

A worked example: Crossplane v2 Namespaced Control Plane

In Crossplane v2, platform APIs can be defined with namespaced scope (scope: Namespaced), allowing product teams to create composite resources directly inside their team namespaces without requiring cluster-wide privileges. Upbound Provider AWS v2 provides matching namespaced managed resources under the *.aws.m.upbound.io API groups.

1. The CompositeResourceDefinition (XRD)

The platform team defines the contract in the cluster:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: databases.platform.example.com
spec:
  group: platform.example.com
  names:
    kind: Database
    plural: databases
  scope: Namespaced
  claimNames:
    kind: DatabaseClaim
    plural: databaseclaims
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: [size]
              properties:
                size:
                  type: string
                  enum: [small, medium, large]
                engineVersion:
                  type: string
                  enum: ["15", "16", "17"]
                  default: "16"
                tier:
                  type: string
                  enum: [development, production]
                  default: development
            status:
              type: object
              properties:
                endpoint:
                  type: string
                port:
                  type: integer
                secretName:
                  type: string
                ready:
                  type: boolean

2. The Composition with Complete Supporting Resources

The platform team writes a Composition that synthesizes the entire infrastructure graph shown in the architecture diagram: KMS encryption key, Security Group, Subnet Group, Parameter Group, RDS Instance, and exported credentials secret.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: database-aws-postgresql
  labels:
    provider: aws
    engine: postgresql
spec:
  compositeTypeRef:
    apiVersion: platform.example.com/v1alpha1
    kind: Database
  mode: Pipeline
  pipeline:
    - step: render-resources
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            {{ $xr := .observed.composite.resource }}
            {{ $name := $xr.metadata.name }}
            {{ $ns := $xr.metadata.namespace }}
            {{ $prod := eq $xr.spec.tier "production" }}
            {{ $sizes := dict "small" "db.t4g.medium" "medium" "db.r7g.large" "large" "db.r7g.2xlarge" }}
 
            ---
            # 1. KMS Encryption Key (Namespaced v2 API)
            apiVersion: kms.aws.m.upbound.io/v1beta1
            kind: Key
            metadata:
              name: {{ $name }}-key
              namespace: {{ $ns }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: kms-key
            spec:
              forProvider:
                description: "KMS storage encryption key for {{ $name }}"
                deletionWindowInDays: {{ if $prod }}30{{ else }}7{{ end }}
                enableKeyRotation: true
                tags:
                  Environment: {{ $xr.spec.tier }}
                  ManagedBy: crossplane
 
            ---
            # 2. Database Security Group (Namespaced v2 API)
            apiVersion: ec2.aws.m.upbound.io/v1beta1
            kind: SecurityGroup
            metadata:
              name: {{ $name }}-sg
              namespace: {{ $ns }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: db-sg
            spec:
              forProvider:
                description: "Ingress security group for PostgreSQL {{ $name }}"
                vpcIdSelector:
                  matchLabels:
                    role: platform-vpc
 
            ---
            # 3. DB Subnet Group (Namespaced v2 API)
            apiVersion: rds.aws.m.upbound.io/v1beta1
            kind: SubnetGroup
            metadata:
              name: {{ $name }}-subnet-group
              namespace: {{ $ns }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: db-subnet-group
            spec:
              forProvider:
                description: "Subnets allocated for {{ $name }}"
                subnetIdSelectors:
                  - matchLabels:
                      tier: private-database
 
            ---
            # 4. PostgreSQL Parameter Group (Namespaced v2 API)
            apiVersion: rds.aws.m.upbound.io/v1beta1
            kind: ParameterGroup
            metadata:
              name: {{ $name }}-pg
              namespace: {{ $ns }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: db-parameter-group
            spec:
              forProvider:
                family: "postgres{{ $xr.spec.engineVersion }}"
                description: "Optimized PostgreSQL {{ $xr.spec.engineVersion }} parameters"
                parameter:
                  - name: "log_connections"
                    value: "1"
                  - name: "rds.force_ssl"
                    value: "1"
 
            ---
            # 5. RDS Instance (Namespaced v2 API)
            apiVersion: rds.aws.m.upbound.io/v1beta1
            kind: Instance
            metadata:
              name: {{ $name }}-instance
              namespace: {{ $ns }}
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: db-instance
            spec:
              forProvider:
                region: eu-west-1
                engine: postgres
                engineVersion: {{ $xr.spec.engineVersion | quote }}
                instanceClass: {{ index $sizes $xr.spec.size }}
                allocatedStorage: {{ if $prod }}100{{ else }}20{{ end }}
                maxAllocatedStorage: {{ if $prod }}1000{{ else }}100{{ end }}
                multiAz: {{ $prod }}
                backupRetentionPeriod: {{ if $prod }}14{{ else }}1{{ end }}
                storageEncrypted: true
                kmsKeyIdSelector:
                  matchControllerRef: true
                dbSubnetGroupNameSelector:
                  matchControllerRef: true
                vpcSecurityGroupIdSelectors:
                  - matchControllerRef: true
                parameterGroupNameSelector:
                  matchControllerRef: true
                deletionProtection: {{ $prod }}
                skipFinalSnapshot: {{ not $prod }}
                autoMinorVersionUpgrade: true
                publiclyAccessible: false
              writeConnectionSecretToRef:
                name: {{ $name }}-db-conn
                namespace: {{ $ns }}
 
    - step: automatically-detect-readiness
      functionRef:
        name: function-auto-ready

3. The Developer’s Self-Service Request

Developers submit an intent-driven resource in their assigned namespace:

# xr.yaml
apiVersion: platform.example.com/v1alpha1
kind: Database
metadata:
  name: orders
  namespace: payments
spec:
  size: small
  tier: production
  engineVersion: "16"

Specifying tier: production silently provisions a Multi-AZ cluster, 14 days of encrypted point-in-time recovery, dedicated KMS key rotation, deletion protection, and parameter-enforced SSL. The developer never configures IAM roles, VPC subnet IDs, or cloud passwords.

4. Offline Verification with crossplane render

Compositions can be verified offline in CI before deploying to a control plane. Create a functions.yaml declaration:

# functions.yaml
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-go-templating
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.4.1
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-auto-ready
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-auto-ready:v0.2.1

Run the offline render command:

crossplane render xr.yaml composition.yaml functions.yaml

The CLI evaluates the templating pipeline locally, validates schema constraints, and prints the 5 generated AWS managed resources (Key, SecurityGroup, SubnetGroup, ParameterGroup, Instance) with the targeted credentials secret.

Composition functions

A Composition is a pipeline of functions, each a small program that receives the desired state and returns a modified one.

FunctionWrite logic inUse for
Patch and transformDeclarative patchesSimple field mapping
Go templatingGo templatesConditionals and loops, familiar to Helm users
KCL, CUE, PklA configuration languageTyped, testable configuration logic
Python, Go, TypeScript SDKsA general-purpose languageComplex logic, external lookups
Auto-ready, status, sequencer—Readiness, status propagation, ordering

Start with templating or patch-and-transform. Reach for a real language only when the logic demands it — and when it does, unit-test it.

Tenancy and guardrails

Crossplane’s providers hold powerful cloud credentials. Anyone who can create an object that Crossplane acts on can make it create infrastructure.

  • Developers get RBAC on your XR kinds only, in their own namespaces — never on raw managed resources such as Instance or Role. See RBAC.
  • Namespaced XRs give natural per-team isolation and work with ordinary Kubernetes tooling.
  • One ProviderConfig per environment or account, selected by the Composition, not by the developer.
  • Admission policy for what the schema cannot express: quotas, naming, which tiers a namespace may request — policy management.
  • deletionPolicy: Orphan and deletion protection on anything stateful. Deleting a namespace must not delete a production database.
  • Scope and bound the provider’s IAM role — IAM.

Evolving an API

Your XRD is an API with consumers, and deserves the same care as any other — release and versioning.

  • Additive changes are safe: a new optional field with a default.
  • Breaking changes need a new version (v1alpha1 → v1beta1), both served for a migration period.
  • Changing a Composition changes every instance. Roll out with composition revisions: pin existing resources to the current revision, try the new one on a few, then promote. A careless edit can trigger replacement of live databases across the organisation.
  • Test compositions in CI: crossplane render produces the output for a given input offline, so you can diff it and assert on it before anything reaches a cluster.
  • Package and version XRDs, Compositions and functions as a Configuration, and deliver them through GitOps like any other artifact.

When to choose something simpler

Crossplane is powerful and has a real learning curve and operational weight.

SituationConsider instead
A few infrastructure types that rarely changeTerraform modules in a pipeline
AWS only, and resources already have Kubernetes controllersACK with kro
The abstraction is only Kubernetes objectsA Helm chart with a small values schema, or kro
You need a reviewed plan before every infrastructure changeTerraform or OpenTofu
A small team with no one to own a control planeA managed PaaS

It earns its place when many teams need self-service infrastructure with centrally enforced standards, across accounts or clouds, with continuous reconciliation. The comparison table is in Crossplane on EKS.

Operating it

  • The control-plane cluster is now critical infrastructure. Back it up, monitor provider health, and alert on resources that stay unsynced.
  • Install only the provider families you use; CRD count affects API server memory.
  • Debug top-down: crossplane beta trace database orders -n payments shows the whole tree and where it is stuck.
  • Surface failures to developers in the XR’s status and events — they cannot and should not read provider logs.

Further reading

Graph View

Backlinks