Deploy Self Hosted Runners to a Kubernetes Cluster for Github Actions

What is a runner? There's a self-hosted one? Why would i want it on a kubernetes cluster?

Github runners are machines that run the steps defined in your github action workflow - they're machines that basically executes the commands on your behalf. By default the repositories run these steps on an virtual machine hosted by Github. more about that here

A Self Hosted runner is a machine you own or control that can run the steps in a build workflow on github actions. you can get this by running steps provided by github that attach that machine and make it available in your github account. you can attach a github runner on repository(repo) level and on organization(org) level - if you want one self hosted runner available to several repos in your org. documentation

Having a self hosted runner on kubernetes adds the flexibility to launch, stop and scale these runners.

How to Deploy on Kubernetes

Prerequisites

A github account - of course
A kubernetes cluster -
A github personal access token. how to create
- required permissions for personal access token / or any token. permissions
[optional] An private ssh key for github access. add ssh key

Steps

clone the following repo - github.com/nebed/self-hosted-runners-onk8s - git clone https://github.com/nebed/self-hosted-runners-onk8s.
Enter into directory for cloned repo - cd self-hosted-runners-onk8s

Edit files to match github repository or organization you're deploying to.

Edit entrypoint.sh if you are using an org, comment the first GITHUB_ENDPOINT and uncomment the ones for an org

#!/bin/sh
GITHUB_ENDPOINT="${GITHUB_OWNER}/${GITHUB_REPOSITORY}"
#for organizations
#GITHUB_ENDPOINT=organization_name
registration_url="https://api.github.com/repos/${GITHUB_ENDPOINT}/actions/runners/registration-token"
#for organizations
#https://api.github.com/orgs/${GITHUB_ENDPOINT}/actions/runners/registration-token
echo "Requesting registration URL at '${registration_url}'"

remove ssh configuration if you don't need it delete this 👇 entire block from the file

mkdir $HOME/.ssh
touch $HOME/.ssh/agent-environment
touch $HOME/.ssh/known_hosts
sudo cp /data/secrets/id_rsa /home/github/.ssh/id_rsa
sudo chown -R github:github /home/github/.ssh/id_rsa
cat > .bashrc <<'SH'
SSH_ENV="$HOME/.ssh/agent-environment"

function start_agent {
    echo "Initialising new SSH agent..."
    /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
    echo succeeded
    chmod 600 "${SSH_ENV}"
    . "${SSH_ENV}" > /dev/null
    /usr/bin/ssh-add /home/github/.ssh/id_rsa;
}

# Source SSH settings, if applicable

if [ -f "${SSH_ENV}" ]; then
    . "${SSH_ENV}" > /dev/null
    #ps ${SSH_AGENT_PID} doesn't work under cywgin
    ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
        start_agent;
    }
else
    start_agent;
fi
SH
ssh-keyscan -H github.com >> $HOME/.ssh/known_hosts

Edit deployment.yaml file and update the spec portion with environment variables

containers:
  - name: github-runner
    image: {{docker-repo}}/k8s-github-runner:latest
    imagePullPolicy: Always
    env:
    - name: GITHUB_OWNER
      value: ""
    - name: GITHUB_REPOSITORY
      value: ""
    - name: GITHUB_PAT
      valueFrom:
        secretKeyRef:
          name: github-pat
          key: pat
    volumeMounts:
    - name: git-private-key
      mountPath: "/data/secrets/"
  volumes:
  - name: git-private-key
    secret:
      secretName: github-private-key
      defaultMode: 0600

Delete this portion if you deleted ssh key portion of entrypoint.sh

    volumeMounts:
    - name: git-private-key
      mountPath: "/data/secrets/"
  volumes:
  - name: git-private-key
    secret:
      secretName: github-private-key
      defaultMode: 0600

Build Docker Image replace {{ docker repo }} with your docker repo. and ensure you're logged in to docker

docker login
docker build . -t {{ docker repo }}/k8s-github-runner:latest
docker push {{ docker repo }}/k8s-github-runner:latest

Create kubernetes Namespace 'github' kubectl create ns github

Create kubernetes Secrets for github PAT token and ssh key

kubectl create secret generic github-pat --from-literal='pat={{ private access token }}' -n github
kubectl create secret generic github-private-key --from-file=id_rsa={{ github private key for cicd-buildbot }} -n github

Create Kubernetes deployment kubectl apply -f deployment.yaml

How to: GitHub Actions Self-Hosted Runner on Kubernetes

What is a runner? There's a self-hosted one? Why would i want it on a kubernetes cluster?

How to Deploy on Kubernetes

Prerequisites

Steps