Deploying Bors-NG in production using Docker - Building a release seems to hardcode configuration

I’m looking into deploying Bors-NG for use in my company’s private repositories. Since I don’t currently have any Erlang/Elixir deployments, I thought using Docker would be an easier way to get going. The source repository contains a development Dockerfile, so I thought I would start there and adapt it for production settings.

Unfortunately, I ran into an issue when using mix release to build a standalone application, so that I can generate a final image without the Elixir/NodeJS runtimes. The built configuration seems to contain hardcoded values of the environment variables at the time of build, instead of looking them up at runtime as I expected.

Here is the (two-stage) Dockerfile I currently have:

ARG ELIXIR_VERSION=1.4.5

FROM elixir:${ELIXIR_VERSION} as builder

ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

RUN apt-get update -q && apt-get install -y build-essential libtool autoconf curl

RUN DEBIAN_CODENAME=$(sed -n 's/VERSION=.*(\(.*\)).*/\1/p' /etc/os-release) && \
    curl -q https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add - && \
    echo "deb http://deb.nodesource.com/node_8.x $DEBIAN_CODENAME main" | tee /etc/apt/sources.list.d/nodesource.list && \
    apt-get update -q && \
    apt-get install -y nodejs

RUN mix local.hex --force && \
    mix local.rebar --force && \
    mix hex.info


WORKDIR /app
ADD ./bors-ng/ /app/

# Set default environment for building
ENV MIX_ENV=prod
ENV GITHUB_CLIENT_ID= GITHUB_CLIENT_SECRET= GITHUB_INTEGRATION_ID=0 GITHUB_INTEGRATION_PEM=
ENV DATABASE_URL=
ENV PORT=9999 PUBLIC_HOST=localhost SECRET_KEY_BASE= GITHUB_WEBHOOK_SECRET=

RUN mix deps.get
RUN cd /app/apps/bors_frontend && npm install && npm run deploy
RUN mix release --env=$MIX_ENV

####

FROM debian:jessie-slim

RUN apt-get update -q && apt-get install -y --no-install-recommends git-core libssl1.0.0

ENV LANG=C.UTF-8
WORKDIR /app
COPY --from=builder /app/_build/prod/rel/ /app/

CMD ["./bors_frontend/bin/bors_frontend", "foreground"]

Here is sys.config as generated at first run. Notice how some variables are left to be interpolated at runtime, and others have already been replaced:

%% Generated - edit/create /app/bors_frontend/sys.config instead.
[{sasl,[{errlog_type,error}]},
 {bors_database,
     [{ecto_repos,['Elixir.BorsNG.Database.Repo']},
      {'Elixir.BorsNG.Database.Repo',
          [{adapter,'Elixir.Ecto.Adapters.Postgres'},
           {url,<<>>},
           {pool_size,10},
           {ssl,true},
           {loggers,[{'Elixir.Ecto.LogEntry',log,[]}]}]}]},
 {wobserver,
     [{mode,plug},
      {security,'Elixir.BorsNG.WobserverSecurity'},
      {remote_url_prefix,<<"/wobserver">>},
      {security_key,
          <<107,174,131,109,206,203,6,74,170,227,134,12,103,79,20,170,8,133,
            92,143,161,31,0,9,97,245,65,49,197,208,56,146,135,116,103,245,
            55,202,156,161,188,135,203,114,77,40,124,193,62,91,10,21,127,8,
            69,164,24,39,157,31,46,205,179,241,253,102,232,32,39,0,195,180,
            220,45,28,57,70,172,252,48,140,12,48,134,82,181,100,203,30,55,
            147,118,24,127,32,81,185,215,225,140,210,153,68,202,159,155,68,
            249,18,130,31,0,82,221,22,228,92,54,227,80,252,207,65,17,33,88,
            59,122>>}]},
 {bors_frontend,
     [{ecto_repos,[]},
      {'Elixir.BorsNG',
          [{command_trigger,<<"bors">>},
           {home_url,<<"https://bors.tech/">>},
           {allow_private_repos,false}]},
      {'Elixir.BorsNG.Endpoint',
          [{render_errors,
               [{view,'Elixir.BorsNG.ErrorView'},
                {accepts,[<<"html">>,<<"json">>]}]},
           {pubsub,
               [{name,'Elixir.BorsNG.PubSub'},
                {adapter,'Elixir.Phoenix.PubSub.PG2'}]},
           {http,[{port,{system,<<"PORT">>}}]},
           {url,
               [{host,{system,<<"PUBLIC_HOST">>}},
                {scheme,<<"https">>},
                {port,443}]},
           {check_origin,false},
           {cache_static_manifest,<<"priv/static/manifest.json">>},
           {secret_key_base,<<>>}]},
      {'Elixir.BorsNG.WebhookParserPlug',[{webhook_secret,<<>>}]}]},
 {bors_github,
     [{site,<<"https://api.github.com">>},
      {server,'Elixir.BorsNG.GitHub.Server'},
      {oauth2,'Elixir.BorsNG.GitHub.OAuth2'},
      {'Elixir.BorsNG.GitHub.OAuth2',[{client_id,<<>>},{client_secret,<<>>}]},
      {'Elixir.BorsNG.GitHub.Server',[{iss,0},{pem,<<>>}]}]},
 {bors_worker,[{ecto_repos,[]}]},
 {logger,
     [{level,info},
      {console,
          [{format,<<"$time $metadata[$level] $message\n">>},
           {metadata,[request_id]}]}]}].

Apologies if I’m committing any obvious mistakes, as I don’t have any production experience with Elixir apps.

1 Like

I ran into the same problem; this is an issue with how the config file works in Bors-NG. There was a little bit of conversation on Gitter about this: https://gitter.im/bors-ng/Lobby?at=5a16c073540c78242d3c8433

1 Like

Working on it.

1 Like

@mithrandi @danielkza Okay, that pull request has now been pushed to master. Does it help?

Thank you!I can successfully build a release and have the app start from it and connect to the database. I’m working on a way to have it create the DB structure and migrate the DB at startup, since Mix tasks can’t be run from releases. But I think I’m close to getting it working nicely.

1 Like

The other pull request has been merged, too. Thanks @danielkza!

And now, the builds are being automatically pushed to the docker hub.

docker create --name bors --restart=unless-stopped \
    -e PUBLIC_HOST=app.bors.tech \
    -e SECRET_KEY_BASE=<secret> \
    -e GITHUB_CLIENT_ID=<secret> \
    -e GITHUB_CLIENT_SECRET=<secret> \
    -e GITHUB_INTEGRATION_ID=<secret> \
    -e GITHUB_INTEGRATION_PEM=<secret> \
    -e GITHUB_WEBHOOK_SECRET=<secret> \
    borsng/bors-ng
docker start bors

Have fun! :bors: :+1:

Unfortunately I’m unable to connect to my database using the borsng/bors-ng image:

17/01/2018 10:04:05{"init terminating in do_boot",{#{'__exception__'=>true,'__struct__'=>'Elixir.RuntimeError',message=><<"The database for BorsNG.Database.Repo couldn't be created: ssl not available">>},[{'Elixir.BorsNG.Database.Migrate',create_storage_for,1,[{file,"lib/migrate.ex"},{line,41}]},{'Elixir.BorsNG.Database.Migrate','-up/1-fun-0-',1,[{file,"lib/migrate.ex"},{line,26}]},{'Elixir.Enum','-each/2-lists^foreach/1-0-',2,[{file,"lib/enum.ex"},{line,645}]},{'Elixir.Enum',each,2,[{file,"lib/enum.ex"},{line,645}]},{'Elixir.BorsNG.Database.Migrate',run_standalone,0,[{file,"lib/migrate.ex"},{line,17}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}

Anyone know if there’s a way around this? In my own image it looks like I just edited ssl to False in apps/bors_database/config/prod.secret.exs.

EDIT: Ah, I see that’s an envvar now; trying to set this at runtime to see if that fixes it.

Yay, after changing that, and granting the database user privs to create databases (not ideal, but probably fine in my case), everything works on the “official” image.

1 Like