Week 9 - Moving to Kubernetes¶

Welcome to the start of phase 3!

This week you will start migrating your application from a virtual machine to a single namespace Kubernetes deployment. You'll need to build container images, access the student Kubernetes cluster, and start working on the Kubernetes deployment. We recommend you to do the initial deployment manually from your personal computer with kubectl commands and manifest files to get a feel for it at first.

Development requirements¶

There are no new development requirements this week. Ensure all features from previous phases continue to work as expected after deploying your application to Kubernetes.

Operations requirements¶

Building container images¶

Configure your GitLab CI/CD pipeline to:

• Build container images for your application.
• Push images to your GitLab project registry.

We recommend using Buildah or BuildKit to build and push containers inside the pipeline.

Accessing the student Kubernetes cluster using ETAIS auth¶

Kubernetes allows access via the MyAccessID authentication system. This is the easiest way to obtain access to the cluster, as everyone shares the same KUBECONFIG file.

Authenticating via MyAccessID requires completing three additional steps:

• Install the kubelogin kubectl plugin. This is required to authenticate with MyAccessID.
• Add the shared KUBECONFIG file to your local computer (example below). Add it to your ~/.kube/config, which is automatically used for all kubectl commands.
• When using this configuration for kubectl, the first command you enter opens a browser window.
• Inside this browser window you can log in with your University of Tartu credentials. Upon success, you'll have access to the cluster.

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJT0lHaVd0ZHQvTDh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TlRBNU1Ea3hNakUwTlRaYUZ3MHpOVEE1TURjeE1qRTVOVFphTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUM3OVkzY0wzM2ozSjg0bDZhS0FseEtnVkZMemVLeTZFdnpGSmNhVWtWODlEQStXWlJ6RWdXenQ0aUcKcWNobjZ2UEZlWVBVSWhhQlErYzkweTlaN01nY0ZSMThqRkZKVWxpSndpRXI0VnBmSjZZNzZzNTRzdVhMY1pPSwo0U0k1VW5LM2l4cnM0UkRtR0lVeXlKVndDcndNWkZ6YmlKUnJEa3RsN2ZQOFRMYytqYyt2MWpDT2F2ZUJ2ejZSCjh4M1RYVVZoY2hsYS8wMkNDTjQwRjMrRnVJa2sxbzJvMnVDV2UxQk90MS9xMzlsVXhFTnlyNnpvOVF6WG9mUkwKdjN2NDJSc3hZZC9qd2dQbDJhQVUzUlNWVVdkU0w5V1BmZzI0bFVuYVdqdG1pMytPTDkybEJobjFKOS9QaUZYdAppUVg0TEJ0WEdLSnIwK3VoblRUSHVkSXZKa1dIQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJSOGRwNE1UVktUTnBUcWNkS0tpdFdDeEZJdVZEQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQlRGQmlXL0NiMAprYU5hbzFqMzR5RXBWd1ZjSmgrTGNKYVN1WUFENUV1Y1gxYkIzb0ZhOXZCczFxR2MrRmwrNWRPRm41R3BoQWRlCkZXYXpWcUtwdkVkZmVFWUJIQkc3akRHdDdaNDFubk5ueC9FeVdUa3QyeG9icmE1NFQ1ODk1cSsrbCt2VjNZU08KaUJhUmZ5WjVZdTVGOEVPbkw0MU9uRzd3VXJUVE9HMzkwanIybnY4TWpOeU8rTExtUzRSNW1taWszVTlBTU04QQpDUUtRMVhnSVZadGlYd1pLR1lzc0dmQW92RHY5aHdXbmJrWm81d0FHSFdDaE9RcEtkZFNqNDZvQ1pQb1NaTzNoCnFpNVNTVVMxM0d4S0hpUGVNQXA3Zi8wUE1ONUhyUGJYMTRwM20xTXhKM2RlSkR5QUhQQ2Rzb0dGUlNJaHA3L0MKb1JkaTdRWmN1VVN4Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: https://kubernetes.devops.cs.ut.ee:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: etais-user
  name: etais-user@kubernetes
current-context: etais-user@kubernetes
kind: Config
preferences: {}
users:
- name: etais-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - oidc-login
      - get-token
      - --oidc-issuer-url=https://keycloak.hpc.ut.ee/realms/ETAIS
      - --oidc-client-id=kubernetes.devops.cs.ut.ee
      - --oidc-pkce-method=auto
      command: kubectl
      env: null
      provideClusterInfo: false

Namespace¶

One namespace has been created for every team and all of the teams members have been given access. The naming of the namespace follows these rules:

• all letters are lower case,
• all spaces and _ have been replaced with -.
• -staging has been added to the end of the team name.

For example, if the teams name is DevOps Team_11 then the namespace would be devops-team-11-staging.

List all resources deployed in the namespace with the command kubectl get all -n <your-team-namespace>. The -n flag is important for all future kubectl commands. Or you could use kubectl context management to do less typing.

Example deployment and testing¶

A Deployment manages a set of Pods to run and application workload.

The following is an example of a Deployment, this is just so you get an idea what happens inside Kubernetes. The Deployment creates a ReplicaSet to bring up three nginx Pods:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: <your-team-namespace>
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

Save the file as nginx-deployment.yaml, change the namespace and deploy it to your cluster using the command kubectl apply -f nginx-deployment.yaml. Run kubectl get deployments to check if the Deployment was created. READY displays how many replicas of the application are available to your users. To see the ReplicaSet created by the Deployment, run kubectl get rs. And finally to see the pods generated run kubectl get pods. If all the Pods are in Ready status then the Deployment is fully deployed.

To get the full information about the deployment run kubectl describe deployments. If there are issues starting the app, under Events one might find the reason why. To get the logs produced by the container run kubectl logs pod/<pod-name>.

Use port forwarding to connect to the nginx server running. Try kubectl port-forward deployment/nginx-deployment -n <your-team-namespace> 4444:80. Now when you go to the address localhost:4444 in your browser you should see a nginx welcome page.

Kubernetes Documentation is your bible now. There are thorough examples on all the resources with explanations. Use it!

Manual Kubernetes deployment¶

Workflow suggestions:

1. Build your application into containers, run on virtual machine first.
2. Push to a registry (registry.hpc.ut.ee this week, GitLab later)
   • First, use public projects, so there’s no authentication issues when Kubernetes is trying to access.
3. Write your deployment files, apply to the cluster. See if containers themselves start.
4. Use emptyDir volumes for storage. We'll cover persistent volumes in the 3rd week of this phase.
5. Start finding a way to pull images from private registries. Start building the images using CI/CD.

Registry access configuration

You will need to give your Kubernetes namespace access to your GitLab container registry to retrieve your application images. This needs to be done using a secret as making your application images public is generally not a good idea.

To pull images from private GitLab registry, you will need to:

• Create a deploy token with read_registry permissions.
• Create a secret in Kubernetes with the generated credentials.
• Reference that secret in manifest spec using imagePullSecrets.

Deployment

Create Kubernetes manifest files to define your application's desired state. Use kubectl commands to apply your manifest files. In phase 3 your application should run with only one replica. Replication we'll do in the next phase.

kubectl apply -f deployment.yaml -n <your-team-namespace>

Testing

Use port forwarding to verify your pods are serving your application correctly. Port forwarding creates a secure tunnel from your local machine to a pod running in the cluster.

kubectl port-forward TYPE/NAME [options] -n <your-team-namespace>  LOCAL_PORT:REMOTE_PORT

We'll set up proper ingress controllers and services for external access next week.

Tips for debugging: use Lens to access logs quickly and to visualize resources. Basic tutorial here.

Lens might not work with default setting. Go to the cluster settings and navigate to the Metrics section. Change the metrics source to No metrics and press the Hide all metrics button. Then reconnect to the cluster manually. Now Lens should display your pods etc.

Documentation requirements¶

Tasks¶

• Build application container images in GitLab CI/CD and push them to your GitLab project container registry.
• Make sure you have access to a namespace in our Kubernetes cluster.
• Give your namespace access to your private GitLab project container registry.
  • You need to use a secret!
• Deploy your application to Kubernetes using kubectl and manifest files.
  • Pull the image from the GitLab container registry.
  • In phase 3 app runs with 1 replica.
• Use port forwarding to test your application pods.
• Start phasing out all VM-related deployment configurations and processes.