Modelplane Modelplane docs
Version
main v0.2 v0.3 v0.4 Latest

ServingStack Custom Resource

On this page

A ServingStack installs the serving substrate on a Kubernetes cluster: LeaderWorkerSet (Standard) or Grove and the KAI Scheduler (Dynamo), plus Gateway API, cert-manager, and Prometheus.

#Metadata

API version
infrastructure.modelplane.ai/v1alpha1
Kind
ServingStack
Scope
Namespaced
Short names
ss

#Example

Manifest
apiVersion: infrastructure.modelplane.ai/v1alpha1
kind: ServingStack
metadata:
  name: west-gke-stack
  namespace: platform
spec:
  cloud: GKE
  secrets:
    - type: Kubeconfig
      name: west-gke-kubeconfig
      key: kubeconfig
    - type: GoogleApplicationCredentials
      name: west-gke-sa-key
      key: private_key
  gateway:
    listeners:
      - name: http
        port: 80
        protocol: HTTP

#Spec

ServingStackSpec defines the desired state of ServingStack.

# cloud required enum: GKE | EKS | AKS | Nebius | Vultr | Existing

The cloud the target cluster runs on. Selects the fixed set of components and versions this stack installs there, which is resolved per cloud at build time and changes only with a Modelplane release. Mirrors InferenceCluster.spec.cluster.source; the cluster composition sets it.

# gateway optional object

Configuration for the cluster’s inference traffic gateway.

# className optional string 1–63 chars default: envoy

GatewayClass name. Override if the cluster already has a GatewayClass named envoy.

# clientCAs optional object[] ≤ 32 items
# certificate required string 1–16384 chars

The CA certificate, PEM encoded.

# name required string 1–253 chars

The InferenceGateway this CA belongs to.

# hostname optional string 1–253 chars

The name this cluster’s gateway is reached by, projected from the InferenceCluster. The gateway serves a certificate for it, so an InferenceGateway can originate TLS and know it reached the right cluster. Without it the gateway serves plain HTTP and carries no traffic, since an InferenceGateway addresses a cluster by name.

# listeners optional object[] ≤ 8 items
# name required string 1–63 chars

Unique listener name.

# port required integer 1–65535

Port number for this listener.

# protocol required enum: HTTP | TCP

Protocol for this listener.

# secrets required object[] 1–8 items
# key required string ≤ 253 chars

Key within the Secret that holds the credential data.

# name required string ≤ 253 chars

Name of the Secret.

# namespace optional string ≤ 253 chars

Namespace of the Secret, when it isn’t this ServingStack’s namespace. Set on cloud identity entries whose credential is the Secret the cloud provider’s ProviderConfig references.

# type required enum: Kubeconfig | GoogleApplicationCredentials | AWSWebIdentityCredentials | NebiusServiceAccountCredentials

The type of credential this secret contains. Kubeconfig is required. Any other value is a cloud identity type; when present, the serving stack authenticates to the cluster as that identity instead of using the kubeconfig’s embedded credentials.

# stack optional enum: Standard | Dynamo default: Standard

Which serving stack this installs. Standard (the default) is the Modelplane-composed serving layer: a Deployment or LeaderWorkerSet, Gateway API, and the endpoint picker. Dynamo swaps in NVIDIA’s components: Grove with the KAI Scheduler for multi-node gang scheduling, and a shared ModelExpress server for weight distribution. Propagated from the InferenceCluster.

#Status

# gateway optional object

Status of the cluster’s inference gateway.

# address optional string ≤ 256 chars

The gateway’s external address, once assigned by the cloud load balancer.

# caCertificate optional string ≤ 16384 chars

PEM certificate of the CA that signed this gateway’s serving certificate. An InferenceGateway validates the gateway against it, so it reaches the cluster it meant to and not whatever answers on that address.