cloudnative wiki

Crossplane on EKS

4 min read

Crossplane on EKS

Crossplane turns a Kubernetes cluster into a control plane for infrastructure. You declare an S3 bucket or an RDS instance as a Kubernetes object, and a controller creates it in AWS and keeps it matching the declaration — the same reconcile loop that keeps a Deployment at three replicas, applied to cloud resources.

Its real purpose is one level higher: letting a platform team publish its own APIs (“a PostgresDatabase with a size and a version”) that hide dozens of underlying resources. That makes it a building block for platform engineering.

The pieces

ConceptWhat it is
ProviderA controller package for one API family, e.g. provider-aws-s3, provider-aws-rds
Managed resource (MR)One Kubernetes object per cloud resource — Bucket, Instance, Role
ProviderConfigHow a provider authenticates: which credentials, which account
XRD (CompositeResourceDefinition)The schema of an API you define
CompositionThe implementation of that API: a pipeline of functions that emits managed resources
XR (composite resource)An instance of your API, created by a platform user

Install and authenticate

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm upgrade --install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system --create-namespace

Install only the provider families you need. The AWS provider is split per service precisely because the monolithic one installed nearly a thousand CRDs and strained the API server.

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-s3
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v1
  runtimeConfigRef:
    name: aws-irsa
---
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: IRSA

The provider pod’s service account gets an IAM role through IRSA or Pod Identity. No access keys are stored. For multi-account setups, that role assumes a role in each target account, selected by a per-account ProviderConfig.

The provider’s role is as powerful as the infrastructure it manages. Anyone who can create a managed resource in the cluster can make AWS create it. Scope the role, set a permissions boundary, and control who can create which resource kinds with RBAC and admission policy.

A managed resource

apiVersion: s3.aws.upbound.io/v1beta2
kind: Bucket
metadata:
  name: payments-exports
spec:
  forProvider:
    region: eu-west-1
    tags: { team: payments }
  deletionPolicy: Orphan
  providerConfigRef:
    name: default

kubectl get bucket shows READY and SYNCED. If someone changes the bucket in the console, Crossplane changes it back at the next reconcile. deletionPolicy: Orphan leaves the cloud resource in place when the object is deleted — the safe setting for anything holding data.

Building a platform API

Developers should not write raw managed resources. Define what they may ask for:

apiVersion: apiextensions.crossplane.io/v2
kind: CompositeResourceDefinition
metadata:
  name: databases.platform.example.com
spec:
  scope: Namespaced
  group: platform.example.com
  names: { kind: Database, plural: databases }
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                size: { type: string, enum: [small, medium, large] }
                engineVersion: { type: string, default: "16" }
              required: [size]

A Composition then maps size: small to an instance class, a subnet group, a security group, a parameter group, a KMS key and a Secrets Manager entry, using a pipeline of composition functions (patch-and-transform, Go templates, KCL or Python). A developer’s entire request becomes:

apiVersion: platform.example.com/v1alpha1
kind: Database
metadata:
  name: orders
  namespace: payments
spec:
  size: small

The platform team owns the Composition and can change defaults — encryption, backups, instance generation — for every database at once.

How it compares

CrossplaneACKkroTerraform / OpenTofu
ModelContinuous reconcileContinuous reconcileContinuous reconcilePlan and apply on demand
CloudsManyAWS onlyAny CRDs present in the clusterMany
Custom abstractionsXRDs and CompositionsNoneResourceGraphDefinitionsModules
StateThe Kubernetes API (etcd)The Kubernetes APIThe Kubernetes APIA state file
Preview of changesLimitedNoneNoneplan — its biggest advantage
MaturityHigh, steep learning curvePer-service, AWS-maintainedYoungVery high

A pragmatic split: Terraform for foundations that change rarely and need a reviewed plan (accounts, VPCs, the cluster itself), and a control plane for resources that belong to an application’s lifecycle and are requested self-service.

Operating notes

  • The cluster becomes critical infrastructure. Deleting an XR can delete a database. Back up the cluster’s API objects, use Orphan for stateful resources, and protect namespaces from accidental deletion.
  • CRD count affects API server memory and client discovery time; install the minimum set of providers.
  • Rate limits. Many resources reconciling against the same AWS API get throttled. Tune poll intervals and provider concurrency.
  • Debugging is kubectl describe on the managed resource: the Synced condition carries the AWS error message verbatim. crossplane beta trace <kind> <name> shows the whole tree under an XR.
  • Importing existing resources is done by setting the crossplane.io/external-name annotation, optionally with managementPolicies: ["Observe"] to adopt without changing anything.
  • Deliver XRs and Compositions with Argo CD or Flux, so infrastructure requests are pull requests.

Graph View

Backlinks