How to: GitHub Actions Self-Hosted Runner on Kubernetes
What is a runner? There's a self-hosted one? Why would i want it on a kubernetes cluster?
Github runners are machines that run the steps defined in your github action workflow - they're machines that basically executes the commands on your behalf. By default the repositories run these steps on an virtual machine hosted by Github. more about that here
A Self Hosted runner is a machine you own or control that can run the steps in a build workflow on github actions. you can get this by running steps provided by github that attach that machine and make it available in your github account. you can attach a github runner on repository(repo) level and on organization(org) level - if you want one self hosted runner available to several repos in your org. documentation
Having a self hosted runner on kubernetes adds the flexibility to launch, stop and scale these runners.
How to Deploy on Kubernetes
Prerequisites
- A github account - of course
- A kubernetes cluster -
- A github personal access token. how to create
- required permissions for personal access token / or any token. permissions
- [optional] An private ssh key for github access. add ssh key
Steps
- clone the following repo - github.com/nebed/self-hosted-runners-onk8s -
git clone https://github.com/nebed/self-hosted-runners-onk8s
. - Enter into directory for cloned repo -
cd self-hosted-runners-onk8s
Edit files to match github repository or organization you're deploying to.
Edit entrypoint.sh if you are using an org, comment the first GITHUB_ENDPOINT and uncomment the ones for an org
#!/bin/sh GITHUB_ENDPOINT="${GITHUB_OWNER}/${GITHUB_REPOSITORY}" #for organizations #GITHUB_ENDPOINT=organization_name registration_url="https://api.github.com/repos/${GITHUB_ENDPOINT}/actions/runners/registration-token" #for organizations #https://api.github.com/orgs/${GITHUB_ENDPOINT}/actions/runners/registration-token echo "Requesting registration URL at '${registration_url}'"
remove ssh configuration if you don't need it delete this ๐ entire block from the file
mkdir $HOME/.ssh touch $HOME/.ssh/agent-environment touch $HOME/.ssh/known_hosts sudo cp /data/secrets/id_rsa /home/github/.ssh/id_rsa sudo chown -R github:github /home/github/.ssh/id_rsa cat > .bashrc <<'SH' SSH_ENV="$HOME/.ssh/agent-environment" function start_agent { echo "Initialising new SSH agent..." /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}" echo succeeded chmod 600 "${SSH_ENV}" . "${SSH_ENV}" > /dev/null /usr/bin/ssh-add /home/github/.ssh/id_rsa; } # Source SSH settings, if applicable if [ -f "${SSH_ENV}" ]; then . "${SSH_ENV}" > /dev/null #ps ${SSH_AGENT_PID} doesn't work under cywgin ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || { start_agent; } else start_agent; fi SH ssh-keyscan -H github.com >> $HOME/.ssh/known_hosts
- Edit deployment.yaml file and update the spec portion with environment variables
Delete this portion if you deleted ssh key portion of entrypoint.shcontainers: - name: github-runner image: {{docker-repo}}/k8s-github-runner:latest imagePullPolicy: Always env: - name: GITHUB_OWNER value: "" - name: GITHUB_REPOSITORY value: "" - name: GITHUB_PAT valueFrom: secretKeyRef: name: github-pat key: pat volumeMounts: - name: git-private-key mountPath: "/data/secrets/" volumes: - name: git-private-key secret: secretName: github-private-key defaultMode: 0600
volumeMounts: - name: git-private-key mountPath: "/data/secrets/" volumes: - name: git-private-key secret: secretName: github-private-key defaultMode: 0600
- Build Docker Image replace {{ docker repo }} with your docker repo. and ensure you're logged in to docker
docker login
docker build . -t {{ docker repo }}/k8s-github-runner:latest
docker push {{ docker repo }}/k8s-github-runner:latest
- Create kubernetes Namespace 'github'
kubectl create ns github
- Create kubernetes Secrets for github PAT token and ssh key
kubectl create secret generic github-pat --from-literal='pat={{ private access token }}' -n github kubectl create secret generic github-private-key --from-file=id_rsa={{ github private key for cicd-buildbot }} -n github
- Create Kubernetes deployment
kubectl apply -f deployment.yaml