kubernetes kustomize

https://github.com/kubernetes-sigs/kustomize

When managing multiple environments, there would be some environment specific that can’t be dealt with only kubernetes ConfigMaps:

• Host for kubernestes ingress
• physical resources affecting kubernetes pods
• metadata (kubernetes label, kubernetes annotation, …) shared to the resources

Kustomize gives the possibility to:

• load classic descriptors
• create kubernetes ConfigMap and kubernetes Secret
• apply transformation

Concept

• no template
• no variable
• static configuration: what is deployed is what is described

Kustomize generators

• configMapGenerator
• secretGenerator

Generate kubernetes ConfigMap and kubernetes Secret from:

• entry key=value
• files
• env variable file (one line == key=value)

Kustomize transformers

• commonAnnotations, comonLabels: global annotations and labels definition
• namespace: global kubernetes namespace definition
• namePrefix, nameSuffix: resource name modification
• images: replace image reference
• patches: apply updates from a “patch”
• custom: we can extend Kustomize with custom transformers

Kustomization project structure

tree ~/someApp/
├── base/
│   ├── deployment.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays/
    ├── development/
    │   ├── kustomization.yaml
    │   └── ingress.yaml
    └── production/
        ├── kustomization.yaml
        ├── memory-limit.yaml
        └── ingress.yaml

Kustomization example

---
# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
nameSuffix: -v1
resources:
  - service.yaml
  - deployment.yaml

Overlay example

---
# overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: prod
resources:
  - ../../base
  - ingress.yaml
patches:
  - path: memory-limit.yaml
configMapGenerator:
  - name: my-config
    literals:
      - ENVIRONMENT=prod

Merge patch example

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  template:
    spec:
    containers:
      - name: my-app
        resources:
          limits:
            memory: 256Mi

Considerations

• declare in advance environment specific in a specific Overlay
• composition between base and overlays at several levels
• define components to compose multiple configurations

Local updates

Useful in continuous integration workflow

# update an image
$ kustomize edit set image \
my.registry.com/myimage=my.registry.com/myimage:${TAG_VERSION}
 
$ # add an annotation
$ kustomize edit add annotation deploy-build-id:${PIPELINE_ID}