Creating Scalable Applications with Kubernetes

📕 Chapter 5: Security, Cost Optimization & Best Practices in Kubernetes

🌐 Introduction

While Kubernetes excels at automating deployment and scaling, security and cost optimization often become afterthoughts — until vulnerabilities are exploited or cloud bills spike.

In this final chapter, we focus on:

• Securing workloads, cluster access, and secrets
• Enforcing pod security and network policies
• Optimizing resource usage and cost
• Implementing architectural and DevOps best practices for production readiness

🔐 Section 1: Kubernetes Security Essentials

Securing a Kubernetes cluster means hardening multiple layers: infrastructure, network, pods, APIs, and CI/CD.

✅ 1.1 Role-Based Access Control (RBAC)

RBAC regulates who can do what.

🔧 Sample Role & RoleBinding

yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: Role

metadata:

  name: pod-reader

rules:

- apiGroups: [""]

  resources: ["pods"]

  verbs: ["get", "watch", "list"]

yaml

kind: RoleBinding

apiVersion: rbac.authorization.k8s.io/v1

metadata:

  name: read-pods

subjects:

- kind: User

  name: dev-user

roleRef:

  kind: Role

  name: pod-reader

  apiGroup: rbac.authorization.k8s.io

✅ 1.2 Pod Security Admission (PSA)

Kubernetes 1.25+ replaces PodSecurityPolicies (PSP) with Pod Security Admission.

Apply via namespace labels:

bash

kubectl label ns dev pod-security.kubernetes.io/enforce=restricted

✅ 1.3 Secrets & ConfigMaps

• Store sensitive data in Secrets
• Avoid hardcoding env vars or secrets in YAML
• Use KMS providers (AWS KMS, Vault, etc.) for encryption

Example Secret:

yaml

apiVersion: v1

kind: Secret

metadata:

  name: db-password

type: Opaque

data:

  password: bXktc2VjcmV0LXBhc3M=  # base64 encoded

✅ 1.4 Network Policies

Restrict traffic between pods and namespaces.

Example: Deny all ingress by default

yaml

apiVersion: networking.k8s.io/v1

kind: NetworkPolicy

metadata:

  name: deny-all

spec:

  podSelector: {}

  policyTypes:

  - Ingress

✅ 1.5 Security Contexts

Run containers with least privilege.

yaml

securityContext:

  runAsUser: 1000

  runAsNonRoot: true

  readOnlyRootFilesystem: true

  allowPrivilegeEscalation: false

💰 Section 2: Cost Optimization Strategies

Cost creeps up when:

• Pods are overprovisioned
• Logs are over-retained
• Orphaned volumes and resources are left behind

🔍 2.1 Set CPU & Memory Limits

Overprovisioning leads to wasted capacity.

yaml

resources:

  requests:

    memory: "256Mi"

    cpu: "200m"

  limits:

    memory: "512Mi"

    cpu: "500m"

📈 2.2 Use Autoscaling

🧹 2.3 Clean Up Unused Resources

Use:

bash

kubectl get all --all-namespaces

kubectl delete pod <pod-name>

kubectl delete pvc <volume-name>

📦 2.4 Optimize Logging & Monitoring

• Set log retention
• Use Fluent Bit or Loki instead of Elasticsearch
• Limit Prometheus scrape targets and intervals

💰 2.5 Spot/Preemptible Instances

In cloud environments:

• Use Spot nodes for stateless workloads
• Use taints/tolerations to control pod placement

🧠 Section 3: Kubernetes Best Practices

🧩 3.1 Namespaces for Organization

Separate workloads by team or environment:

bash

kubectl create namespace dev

kubectl create namespace prod

Use resource quotas to limit usage per namespace.

⚙️ 3.2 Liveness and Readiness Probes

Ensure only healthy pods receive traffic.

yaml

readinessProbe:

  httpGet:

    path: /health

    port: 8080

  initialDelaySeconds: 5

  periodSeconds: 10

🔐 3.3 Use TLS and Secrets for Communication

• Use Ingress with TLS
• Use cert-manager for auto-renewed Let's Encrypt certificates

🧪 3.4 CI/CD & GitOps

Use:

• ArgoCD / Flux for declarative GitOps
• Helm or Kustomize for templating
• kubectl diff in pipelines for previewing changes

🔍 3.5 Enable Audit Logging

Track access and resource changes.

bash

--audit-log-path=/var/log/kubernetes/audit.log

--audit-policy-file=/etc/kubernetes/audit-policy.yaml

🧼 3.6 Delete Completed Jobs & Orphaned PVCs

bash

kubectl delete jobs --field-selector=status.successful=1

kubectl get pvc --no-headers | awk '{print $1}' | xargs kubectl delete pvc

🛡️ Section 4: CI/CD Security Tips

• Use image scanners (Trivy, Clair) in pipelines
• Pin image versions (my-app:v1.2.3)
• Never use :latest in production
• Restrict Dockerfile capabilities (non-root base images)

✅ Summary

Securing and scaling Kubernetes apps requires continuous guardrails, not one-time configurations.

Key takeaways:

• Apply least privilege with RBAC and security contexts
• Encrypt secrets and enforce network isolation
• Set resource limits and autoscale efficiently
• Audit changes and monitor costs proactively
• Leverage tools like ArgoCD, cert-manager, Prometheus for automation

With security-first practices and proactive cost control, your Kubernetes environment becomes resilient, affordable, and production-ready.