How to deploy your own instance of bors-ng with AWS

notriddle · August 18, 2022, 11:04pm

A note about resource requirements

Bors-NG is a chat bot, not a full CI system, so you can probably provision it on the smallest VM you have the ability to use. The public instance of Bors-NG, which is only usable for public repositories, has over twenty thousand repositories connected to it, processes between one and four requests per second, and runs in a Heroku hobby tier dyno for $7 a month.

First step: setting up the GitHub App

To set up a GitHub App, your first step is to register it within GitHub. You'll need to copy a lot of data from here to the docker-compose.yml later.

Go to the Settings area on GitHub, then click New GitHub App. You'll need a domain name to act as the ENDPOINT.

Name	Set it to
GitHub App Name	Set this to something unique and descriptive.
Homepage URL	This can technically be set to anything.
Callback URL	`https://ENDPOINT/auth/github/callback`
Expire authorization tokens	false
Request User authorization during installation	false
Enable Device Flow	false
Setup URL (optional)	Leave this blank
Redirect on update	false
Webhook active	true
Webhook URL	`https://ENDPOINT/webhook/github`
Webhook secret	Set this to something random. Save it.

Permission	Setting
Actions	No access
Administration	No access
Checks	Read & Write
Code scanning alerts	No access
Codespaces	No access
Codespaces lifecycle admin	No access
Codespaces metadata	No access
Codespaces secrets	No access
Commit statuses	Read & Write
Contents	Read & Write
Dependabot alerts	No access
Dependabot secrets	No access
Deployments	No access
Discussions	No access
Environments	No access
Issues	Read & Write
Merge queues	No access
Packages	No access
Pages	No access
Projects	No access
Pull requests	Read & Write
Secret scanning alerts	No access
Secrets	No access
Single file	No access
Webhooks	No access
Workflows	No access

Organization permisssion	Setting
Administration	No access
Blocking users	No access
Events	No access
Members	Read-only
Organization codespaces	No access
Organization codespaces secrets	No access
Organization dependabot secrets	No access
Plan	No access
Projects	No access
Secrets	No access
Self-hosted runners	No access
Team discussions	No access
Webhooks	No access

User permission	Setting
Block another user	No access
Codespaces user secrets	No access
Email addresses	No access
Followers	No access
GPG keys	No access
Gists	No access
Git SSH keys	No access
Interaction limits	No access
Plan	No access
Profile	No access
Starring	No access
Watching	No access

Subscribe to event	Setting
Meta	false
Security advisory	false
Check run	true
Check suite	true
Commit comment	false
Create	false
Delete	false
Fork	false
Gollum	false
Issue comment	true
Issues	false
Label	false
Milestone	false
Member	true
Membership	true
Merge queue entry	false
Organization	true
Page build	false
Public	false
Pull request	true
Pull request review	true
Pull request review comment	true
Pull request review thread	false
Push	false
Release	false
Repository	true
Repository dispatch	false
Star	false
Status	true
Team	true
Team add	false
Watch	false
Workflow dispatch	false
Workflow job	false
Workflow run	false

Your next step will be generating a private key. Generate it, save it, then run this command to turn it into a base64 string:

$ base64 -w0 < my-bors.2022-08-01.private-key.pem

Also generate and save the client secret.

This leaves us with, in total, three GitHub-specific secret values to keep track of:

Webhook secret: used by bors to validate webhooks coming from GitHub. This is the GitHub -> Bors authentication path.
Private key: used by bors to authenticate as a bot with GitHub. This is the Bors -> GitHub authentication path.
Client secret: used for logging into the bors-ng dashboard with your GitHub account. This is the User -> GitHub authentication path.

Second step: setting up your `ENDPOINT` domain name in AWS Cert Manager

You'll need to create this so that webhooks and the dashboard work correctly, and you'll also need to remember the ARN you get at the end.

To do this, go into the cert manager, request a public certificate with your ENDPOINT domain name, note the ARN for later, and wait until it's validated.

Third step: running bors-ng in AWS ECS

This sample file is in the bors-ng repository. Clone it, then let's modify it:

$ git clone https://github.com/bors-ng/bors-ng
$ cd bors-ng/deploy/examples/docker-compose/bors
$ vim docker-compose.yml

Inside that file are several placeholders that need replaced with other values. The bolded parts below are what you need to change:

version: "2"
services:
  web:
    image: borsng/bors-ng
    restart: always
    ports:
      - target: 8000
        x-aws-protocol: http
    environment:
      # You shouldn't have to change these
      PORT: 8000
      DATABASE_USE_SSL: "false"
      DATABASE_AUTO_MIGRATE: "true"
      ALLOW_PRIVATE_REPOS: "true"
      # Replace this with the hostname of your bors instance
      # For example, the public instance is app.bors.tech
      # and cockroachdb runs bors.crdb.io
      PUBLIC_HOST: [registered]
      # Replace this with a random number
      # If you have a bors source checkout, you can run `mix phx.gen.secret` to get one
      SECRET_KEY_BASE: [random]
      # Get these from the GitHub App setup screen under "OAuth credentials"
      GITHUB_CLIENT_ID: [client id]
      GITHUB_CLIENT_SECRET: [client secret]
      # Get this from the GitHub App setup screen under "About"
      GITHUB_INTEGRATION_ID: [app id]
      # Get this by taking the private key and base64-encoding it
      # For example, `openssl base64 -A -e < bors-merge-queue.2018-08-10.private-key.pem`
      # the result will be long, and will end in one or two equal signs
      GITHUB_INTEGRATION_PEM: [base64 value]
      # Replace this with a random number
      # You will also provide this same number to GitHub when you set up the app
      GITHUB_WEBHOOK_SECRET: [random]
      # Make sure the [censored] password here matches the POSTGRES_PASSWORD
      DATABASE_URL: postgres://postgres:[random]@postgres:5432/bors
    depends_on:
      - postgres
  postgres:
    image: postgres:10.5
    restart: always
    volumes:
      - datadb:/var/lib/postgresql/data
    environment:
      # Make sure the [censored] password here matches the one in the DATABASE_URL
      POSTGRES_PASSWORD: [random]
x-aws-cloudformation:
  Resources:
    Web8000TargetGroup:
      Properties:
        HealthCheckPath: /health
        Matcher:
          HttpCode: 200-499
    Web8000Listener:
      Properties:
        Certificates:
          - CertificateArn: [from aws cert manager]
        Protocol: HTTPS
        Port: 443
volumes:
  datadb:
networks:
  default:
    driver: bridge

Once you've subbed in all your censored secrets, set up bors-ng itself by running these commands in the same directory as the docker-compose.yml file:

$ docker context create ecs myecscontext
$ docker context use myecscontext
$ docker compose up

The name of the directory does matter, because that's the default name of the docker-compose app.

This process will create a load balancer for you, but will not set up DNS for it. You'll need to go into the AWS EC2 console, get the DNS name for the load balancer (it'll look something like bors-LoadBa-BLABLABLABLA-1111111111.us-west-2.elb.amazonaws.com), and set up a CNAME connecting it to the public DNS name.

Once that's set up, you should be able to visit https://ENDPOINT from your browser, and configure the bot from there.

Customizing Bors-NG

If you want to make any changes to the bot, which many enterprise deployments do, you'll need to maintain your own soft-fork. To make this work, you'll be making your own image, and changing the image: borsng/bors-ng line in your docker-compose.yml to point at it instead.

Building a container image with GitHub Actions

After making your local GitHub fork of bors-ng/bors-ng (if you don't mind it being public, click the "Fork" button, otherwise you'll need to create a private repository and push a copy of the code to it), add a new workflow file to automatically build the container.

# .github/workflows/docker.yml
#
# The reason why bors-ng doesn't already have a file like this is that it uses the Docker Hub,
# not GitHub's container registry.
name: bors-docker
on:
  push:
    - master

permissions:
  contents: write
  packages: write

jobs:
  build:
    steps:
      - uses: actions/checkout@master
      - uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      # replace [owner] with an actual org or user
      - run: docker build -t ghcr.io/[owner]/bors-ng .
      - run: docker push ghcr.io/[owner]/bors-ng

Also, patch the docker-compose.yml file, and then run docker compose up again:

version: "2"
services:
  web:
    image: ghcr.io/[OWNER]/bors-ng
    restart: always
    ports:
      - target: 8000
        x-aws-protocol: http
# ----------- most of the file is unchanged -----------