How to: GitHub Actions Self-Hosted Runner on Kubernetes

ยท

3 min read

What is a runner? There's a self-hosted one? Why would i want it on a kubernetes cluster?

Github runners are machines that run the steps defined in your github action workflow - they're machines that basically executes the commands on your behalf. By default the repositories run these steps on an virtual machine hosted by Github. more about that here

A Self Hosted runner is a machine you own or control that can run the steps in a build workflow on github actions. you can get this by running steps provided by github that attach that machine and make it available in your github account. you can attach a github runner on repository(repo) level and on organization(org) level - if you want one self hosted runner available to several repos in your org. documentation

Having a self hosted runner on kubernetes adds the flexibility to launch, stop and scale these runners.

How to Deploy on Kubernetes

Prerequisites

  • A github account - of course
  • A kubernetes cluster -
  • A github personal access token. how to create
    • required permissions for personal access token / or any token. permissions
  • [optional] An private ssh key for github access. add ssh key

Steps

  • clone the following repo - github.com/nebed/self-hosted-runners-onk8s - git clone https://github.com/nebed/self-hosted-runners-onk8s.
  • Enter into directory for cloned repo - cd self-hosted-runners-onk8s
  • Edit files to match github repository or organization you're deploying to.

    • Edit entrypoint.sh if you are using an org, comment the first GITHUB_ENDPOINT and uncomment the ones for an org

      #!/bin/sh
      GITHUB_ENDPOINT="${GITHUB_OWNER}/${GITHUB_REPOSITORY}"
      #for organizations
      #GITHUB_ENDPOINT=organization_name
      registration_url="https://api.github.com/repos/${GITHUB_ENDPOINT}/actions/runners/registration-token"
      #for organizations
      #https://api.github.com/orgs/${GITHUB_ENDPOINT}/actions/runners/registration-token
      echo "Requesting registration URL at '${registration_url}'"
      

      remove ssh configuration if you don't need it delete this ๐Ÿ‘‡ entire block from the file

      mkdir $HOME/.ssh
      touch $HOME/.ssh/agent-environment
      touch $HOME/.ssh/known_hosts
      sudo cp /data/secrets/id_rsa /home/github/.ssh/id_rsa
      sudo chown -R github:github /home/github/.ssh/id_rsa
      cat > .bashrc <<'SH'
      SSH_ENV="$HOME/.ssh/agent-environment"
      
      function start_agent {
          echo "Initialising new SSH agent..."
          /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
          echo succeeded
          chmod 600 "${SSH_ENV}"
          . "${SSH_ENV}" > /dev/null
          /usr/bin/ssh-add /home/github/.ssh/id_rsa;
      }
      
      # Source SSH settings, if applicable
      
      if [ -f "${SSH_ENV}" ]; then
          . "${SSH_ENV}" > /dev/null
          #ps ${SSH_AGENT_PID} doesn't work under cywgin
          ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
              start_agent;
          }
      else
          start_agent;
      fi
      SH
      ssh-keyscan -H github.com >> $HOME/.ssh/known_hosts
      
    • Edit deployment.yaml file and update the spec portion with environment variables
      containers:
        - name: github-runner
          image: {{docker-repo}}/k8s-github-runner:latest
          imagePullPolicy: Always
          env:
          - name: GITHUB_OWNER
            value: ""
          - name: GITHUB_REPOSITORY
            value: ""
          - name: GITHUB_PAT
            valueFrom:
              secretKeyRef:
                name: github-pat
                key: pat
          volumeMounts:
          - name: git-private-key
            mountPath: "/data/secrets/"
        volumes:
        - name: git-private-key
          secret:
            secretName: github-private-key
            defaultMode: 0600
      
      Delete this portion if you deleted ssh key portion of entrypoint.sh
          volumeMounts:
          - name: git-private-key
            mountPath: "/data/secrets/"
        volumes:
        - name: git-private-key
          secret:
            secretName: github-private-key
            defaultMode: 0600
      
  • Build Docker Image replace {{ docker repo }} with your docker repo. and ensure you're logged in to docker
docker login
docker build . -t {{ docker repo }}/k8s-github-runner:latest
docker push {{ docker repo }}/k8s-github-runner:latest
  • Create kubernetes Namespace 'github' kubectl create ns github
  • Create kubernetes Secrets for github PAT token and ssh key
    kubectl create secret generic github-pat --from-literal='pat={{ private access token }}' -n github
    kubectl create secret generic github-private-key --from-file=id_rsa={{ github private key for cicd-buildbot }} -n github
    
  • Create Kubernetes deployment kubectl apply -f deployment.yaml

image.png

ย