RELEASE and DEPLOY Phase

Deployment can be of infrastructure or application; however, we should scan our deployment files. We can also add a manual trigger where the pipeline waits for external user validation before proceeding to the next stage, or it can be an automated trigger.

Static scan of Kubernetes manifest file or Helm chart

It is always a good practice to scan your Kubernetes deployment or Helm chart before deploying. We can use Checkov to scans Kubernetes manifests and identifies security and configuration issues. It also supports Helm chart scanning. We can also use terrascan and kubeLinter to scan the Kubernetes manifest.

docker run -t -v $(pwd):/output bridgecrew/checkov -f /output/keycloak-deploy.yml -o json
# For Helm
docker run -t -v $(pwd):/output bridgecrew/checkov -d /output/ --framework helm -o json

Pre-deploy policy check Kubernete manifest YAML file

Kyverno adds an extra layer of security where only the allowed type of manifest is deployed onto kubernetes, otherwise, it will reject or we can set validationFailureAction to audit which only logs the policy violation message for reporting. Kubewarden and Gatekeeper are alternative tools available to enforce policies on Kubernetes CRD.

Here is a simple Kyverno policy to disallow the image’s latest tag.

kube-bench for CIS scan

kube-bench checks whether Kubernetes is deployed securely by running the checks documented in the CIS Kubernetes Benchmark. We can deploy kube-bench as a Job that runs daily and consume its report in CI/CD to pass or fail the pipeline based on the level of severity.

kubectl apply -f eks-job.yaml
kubectl logs kube-bench-pod-name

IaC scanning:

• Checkov, Terrascan, and Kics can be used to scan our Infrastructure code. It supports Terraform, Cloudformation, and Azure ARM resources.

• Terratest can be used to test infrastructure in real-time.

terraform init
terraform plan -out tf.plan
terraform show -json tf.plan | jq '.' > tf.json
checkov -f tf.json

After scanning for Kubernetes deployment and kube-bench we can deploy our application.

gitlab-ci.yaml

gcloud_deploy:
  stage: gkedeploy
  image: google/cloud-sdk
  script:
    - echo "$GCLOUD_SERVICE_KEY" | base64 -d > gcloud-service-key.json
    - gcloud auth activate-service-account --key-file gcloud-service-key.json
    - gcloud config set project mythical-cider-396513
    - gcloud config set container/cluster ghost-cluster
    - gcloud config set compute/zone asia-east1-a
    - gcloud container clusters get-credentials ghost-cluster --zone asia-east1-a --project mythical-cider-396513
    - kubectl apply -f k8s/ghost/ghost-dep.yaml