cloudnative wiki

Platform Engineering & Internal Developer Platforms (IDP)

4 min read

Platform Engineering & Internal Developer Platforms (IDP) 🏗️

Platform Engineering is the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era.

1. The Core Problem: Cognitive Load & “Shadow DevOps”

Before platform engineering, developers were burdened with sprawling operational complexity: writing raw Helm charts, provisioning IAM roles, configuring ingress controllers, and wrestling with CloudFormation/Terraform state files. This created friction, bottlenecks on operations teams, and widespread configuration drift.

Platform engineering solves this by treating the platform as a product, providing paved paths (Golden Paths) that allow developers to self-serve infrastructure and deployment without needing to become cloud architects.

flowchart TD
    subgraph Devs["Product Developers"]
        Dev1["Frontend Engineer"]
        Dev2["Backend Engineer"]
        Dev3["Data Scientist"]
    end

    subgraph PlatformProduct["Internal Developer Platform (IDP)"]
        Portal["Developer Portal\n(Backstage / Port / Internal CLI)"]
        Scaffold["Service Scaffolding\n(Golden Paths & Templates)"]
        ControlPlane["Declarative Control Plane\n(Crossplane / Argo CD / K8s CRDs)"]
    end

    subgraph CloudInfra["Cloud-Native Infrastructure"]
        EKS["Amazon EKS / AKS / GKE"]
        DB["Cloud Databases (RDS / Cloud SQL)"]
        Net["VPC & Ingress Networking"]
        Sec["IAM Roles & Vault Secrets"]
    end

    Devs -->|Self-Service Request (e.g. New Microservice)| Portal
    Portal --> Scaffold
    Scaffold --> ControlPlane
    ControlPlane -->|Provisions & Reconciles| CloudInfra

2. Platform Architecture Layers

An enterprise Internal Developer Platform typically comprises five operational planes:

PlanePurposeKey Technologies
1. Developer Interface PlaneHow developers interact with the platform (UI, CLI, API)Backstage, Port, Cortex, internal CLI (devctl)
2. Integration & Delivery PlaneAutomated build, test, and release orchestrationGitHub Actions, GitLab CI, Argo Workflows
3. Resource Orchestration PlaneTranslates high-level developer intents into infrastructureCrossplane, Terraform Controller, Kratix
4. Runtime & Compute PlaneWhere container workloads and functions runKubernetes (EKS/AKS/GKE), AWS ECS, Serverless
5. Observability & Security PlaneReal-time health, security policies, and cost allocationOpenTelemetry, Prometheus, Kyverno, OpenCost

3. Core Technologies in Modern Platform Engineering

1. Spotify Backstage

An open-source developer portal framework that centralizes:

  • Software Catalog: Single pane of glass tracking ownership, metadata, and dependencies of all services, APIs, and libraries.
  • Software Templates (Scaffolder): One-click creation of new microservices with pre-baked CI/CD, linting, Dockerfile, and monitoring.
  • TechDocs: Markdown documentation rendered next to code.

2. Crossplane: Kubernetes-Native Infrastructure Control Planes

Crossplane extends the Kubernetes API to manage external cloud resources (RDS databases, S3 buckets, IAM roles) using standard Kubernetes manifests. By defining Composite Resource Definitions (XRDs), platform teams publish simplified abstractions (e.g., kind: DatabaseInstance) while hiding complex multi-cloud Terraform underneath.

4. The “Thinnest Viable Platform” (TVP) Approach

Avoid building a monolithic internal platform before understanding team needs. The CNCF Platform Engineering Maturity Model frames platform evolution around observable pain and developer feedback:

  1. Provisional Stage (Paved Roads): Start with Golden Paths implemented as simple, versioned GitHub Actions templates and cookiecutters that solve immediate onboarding and build friction.
  2. Operational Stage (Service Discovery & Ownership): Introduce a central software catalog when discovery pain, unclear service ownership, or untracked cross-service dependencies measurably slow down incident triage and security audits.
  3. Scalable Stage (Declarative Self-Service Control Planes): Decouple infrastructure provisioning through self-service control planes (Crossplane, GitOps, Kubernetes CRDs) when manual operations tickets and configuration drift become quantifiable delivery bottlenecks.
  4. Optimizing Stage (Continuous Product Feedback): Continuously evaluate platform capabilities using developer satisfaction surveys (DevEx), adoption telemetry, and DORA metrics, deprecating unused tooling and refining golden paths.

Notes in this section

Foundations & Strategy

Developer Interfaces & Abstractions

Orchestration, Control Planes & Environments

Operations & Governance

Graph View

Backlinks