How to deploy applications to K8S with Kustomize and ArgoCD

Step 1

Create .drone.yml

kind: pipeline
name: your-project-name

steps:
  - name: push images to harbor.google.com

  - name: kustomize-beta 
    image: docker.io/drone-with-kustomization:latest
    settings:
      repo_url: ssh://git@git.google.com:20022/PROJECT/abc.git
      branch: master
      kustomization: your-app/overlays/beta
      update_image:
        - harbor.google.com/project/your-app-image:${DRONE_COMMIT_BRANCH}-${DRONE_COMMIT_SHA:0:7}
    environment:
      SSH_KEY:
        from_secret: SSH_KEY
    when:
      branch: develop
      event: push
    depends_on:
      - push image to harbor

Create kustomize deployment files

your-app/base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: your-app-deployment
spec:
  replicas: 3
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app: your-app-deployment
  template:
    metadata:
      labels:
        app: your-app-deployment
    spec:
      containers:
        - name: your-app-deployment
          ports:
            - containerPort: 8888
          # wait until pod is ready
          startupProbe:
            httpGet:
              path: /health
              port: 8888
            failureThreshold: 60
            periodSeconds: 3
            successThreshold: 1
          # remove pod from service if unhealthy
          readinessProbe:
            httpGet:
              path: /health
              port: 8888
            periodSeconds: 3
          # restart pod if unhealthy
          livenessProbe:
            httpGet:
              path: /health
              port: 8888
            failureThreshold: 1
            periodSeconds: 10

your-app/base/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml

Create deployment files for the beta environment in the following folder

your-app/overlays/beta/

1. configMap.yml

kind: ConfigMap
metadata:
  name: your-app-configmap
  namespace: your-app-beta
data:
  enableAuth: "true"
  username: "foo"
  pass: "bar"

2. deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: your-app-deployment
  annotations:
    configmap.reloader.stakater.com/reload: "your-app-configmap"
spec:
  template:
    spec:
      containers:
        - name: your-app-deployment
          image: harbor.google.com/project/your-app-image:beta
          env:
            - name: PROFILE
              value: beta
            - name: PASS_DB
              valueFrom:
                secretKeyRef:
                  name: secret-beta-1231235ye
                  key: APP_PASS_DB
            - name: enableAuth
              valueFrom:
                configMapKeyRef:
                  name: your-app-configmap
                  key: enableAuth
            - name: username
              valueFrom:
                configMapKeyRef:
                  name: your-app-configmap
                  key: username
            - name: pass
              valueFrom:
                configMapKeyRef:
                  name: your-app-configmap
                  key: pass

3. ingress.yml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: your-app-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: your-app.google.com
      http:
        paths:
          - path: "/"
            backend:
              service:
                name: your-app-service
                port:
                  number: 9000
            pathType: Prefix

4. kustomization.yml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base/
  - service.yaml
  - ingress.yaml

commonLabels:
  phase: beta

patchesStrategicMerge:
  - deployment.yaml
images:
  - name: harbor.google.com/project/your-app-image
    newTag: develop-12321fc

5. service.yml

apiVersion: v1
kind: Service
metadata:
  name: your-app-service
spec:
  selector:
    app: your-app-deployment
  ports:
    - port: 9000
      targetPort: 8888
      protocol: TCP

How to config AgroCD

1. Install Agro 

$ brew install argoproj/tap/argocd

$ argocd login --grpc-web --sso argocd.google.com

2.Add cluster

$ kubectl create namespace your-app-beta

$ argocd cluster add --system-namespace=your-app-beta your-cluster-name

 

# Check role after add cluster

$ kubectl describe clusterrole argocd-manager-role

$ kubectl describe clusterrolebinding argocd-manager-role-binding

3. Add repositories in Argo settings

https://argocd.google.com/settings/repos

Repository URL 

git@git.goole.com:20022/PROJECT/abc.git

 SSH Private key

--BEGIN OPENSSH PRIVATE KEY
xxxxxx
--END OPENSSH PRIVATE KEY

4. Create projects

https://argocd.google.com/settings/projects

5. Create applications

https://argocd.google.com/applications

Project name: your-project

Cluster: your-cluster-name (https://google.api/v1/k8s/34hj3h8eYUusy893eree)

Namespace: your-app-beta

Repo URL : ssh://git@git.google.com:20022/PROJECT/abc.git

Targe revision: master

Path: your-app/overlays/beta/

6. Config project ArgoCD to manage multiple namespaces / one or multiple clusters

https://argocd.google.com/settings/projects

Add multiple records of the 3 fields:

Server:  https://google.api/v1/k8s/34hj3h8eYUusy893eree

Cluser name: your-cluster-name

Namespace: *your-app-beta

7. Create secret variables for application

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1

kind: Kustomization

 

secretGenerator:

- name: secret-beta

  literals:

  - MONGOOO_PASS=literal_1

  - IAATT_SECRET=literal_2

  - BOOT_CHANNEL_SECRET=literal_3

and run the cmd below

kubectl apply -k . --namespace your-app-beta

References:

https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

https://kubernetes.io/docs/concepts/services-networking/service/

https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/

https://kubernetes.io/docs/concepts/configuration/secret/