KubeCon + CloudNativeCon Europe 2020 Virtual. Day 1

General impression

The organizers did their best to move the conference online. All the presentations are done on their own platform. One funny thing that their video player didn’t have speed controls, so the current solution is to run the

for(v of document.getElementsByTagName("video")) {v.playbackRate = x}

in the JavaScript console.

There is a slack workspace and every talk gets own slack channel so it’s possible to watch the talk later and ask questions during the whole week, which is a nice improvement compared to the on-site event.

On Day 1 I visited two talks

Help! My Cluster Is On The Internet: Container Security Fundamentals

Speakers: Samuel Davidson, Google

This is 101 talk, but what I took from it is a checklist for securing the Kubernetes cluster. It’s pretty opinionated, though.

The security topics go from the very bottom (containers) to the very top (cluster and user).

Workload security – Container

  1. Use a distroless base image – the less tools are in the image, the less attack surface you have. Going to distroless is normally just one line in your Dockerfile. If it suits you, even use the scratch image
  2. Containers should be easy to build and deploy. Most vulnerabilities are found in dependencies, so your typical work is to bump library version, build and deploy new image.
  3. Trust your containers with signatures. Pretty complex topic, the idea is to sign the image (hash the container) and let CI/CD check the signatures before using them. The problem is that you have to build it on your own and the attacker could still manage to access the signing key.

Workload security – Pods

  1. Don’t use hostPath . You can’t fully control the file system behind it, someone can misuse the path you used. Use secrets instead.
  2. Don’t use hostNetwork . Localhost is treated like a trust domain, this way you violate it.
  3. Be conscious of your Pod’s Service Account. If an attacker has access to the pod they could access the secrets of the attached Service Account.
    1. Solution 1: put a pod in a separate namespace
    2. Solution 2: turn automountServiceAccountToken to false
    3. Solution 3: use a blank/special service account

Cluster Security

  1. Keep your cluster up to date. Upgrading cluster manually is tedious. It’s automatic in Azure, Google Cloud etc.
  2. Isolate your cluster from the internet. Hide it behind a VPN, auth-proxy or set up a firewall. VMs should not have public ips
  3. Use Secrets. They are never saved on disk, loaded only when needed etc. But they too are not bulletproof.

User Security

  1. Use RBAC and groups. Avoid assigning users directly to roles, prefer assigning users to groups (easier to manage via AD) and give roles to groups
  2. Use a policy agent to protect your cluster. Admission Controller will allow/deny kubernetes resource requests based on rules. This way you can enforce those rules about not using hostPath, hostNetwork  etc. It can also audit your current resources.

Full list with detailed explanation is here:

https://bit.ly/SamK8sSec

Tutorial: Getting Started With Cloud Native Security

Speakers: Liz Rice (Aqua Security), Michael Hausenblas (AWS)

This was more or less just a live step-by-step through this tutorial on the web.

The first part was about:

  • Using Trivy to scan the images for vulnerabilities
  • Automating it with Starboard plugin (developed by the Aqua Security).
  • Viewing the result in a nice report in Octant.

The second part was about checking if your cluster follows best security practices. This was mostly a demo of the kube-bench tool developed again by the Aqua Security.

The third part was the demonstration of the Kubernetes network policies.