Deploying Concourse with concourse-up

01.01.2020 | Testing Devops Backend | Austin Vance


There are a lot of Continuous integration tools out there, and they all have minor pros and cons but are generally the same. You push some code, that code triggers a build, and that build could trigger others. Sometimes these are called pipelines, but the only actual implementation of a pipeline build system I have seen is Concourse. There will be future posts on what a pipeline is and why it's such a powerful concept not that's not this post.

For this post, you should be up and running with Concourse in AWS with SSL and have your first pipeline in about 45 minutes, and most of that time will be waiting for AWS.

With this guide, you will

  • Prepare AWS
  • Provision valid SSL Certificates for free
  • Deploy Concourse to AWS using concourse-up

You will need:

  • Docker
  • An internet connection
  • An AWS Account with some credit
  • A Domain with access to the DNS

Prepare AWS

You don't want to give concourse-up too many permissions so let's create some restricted keys and use those for our deployment.

First login to AWS and head to the IAM section.

If you're not familiar with AWS' IAM rules I recommend reading up on them. There are ton's of guides out there that can help.

I like to use Groups to manage Permissions so first create a group by going to the Group's section and then Add New Group. After naming your group, you will be prompted to attach security policies.

Add the following:

  • AmazonRDSFullAccess
  • AmazonEC2FullAccess
  • IAMFullAccess
  • AmazonS3FullAccess
  • AmazonVPCFullAccess
  • IAMUserSSHKeys
  • AmazonRoute53FullAccess

You should see your new group with 0 users in the group list.

Now we need to add a user. In the Users section select Add User, name the user, and check the box Programmatic access. On the next section add the User to the group you created above. You don't need any tags so finish building the user.

This next screen is essential When you see Success click Download CSV and put it in a secure place.

Now you're ready to get your SSL Certs ready

Provision valid SSL Certificates

Concourse uses SSL for all of it's service to service communication. You also want to make sure that any content you serve is also SSL so let's do that next.

Certbot has a ton of options and can be really easy depending on your registrar and where you manage your DNS.
The most basic way is to use Certbot's manual image to generate your certificates.

sudo docker run -it --rm --name certbot \
    -v "$HOME/letsencrypt:/etc/letsencrypt" \
    -v "$HOME/letsencrypt:/var/lib/letsencrypt" \
    certbot/certbot certonly \

Follow the prompts. You will be required to verify ownership of your domain. If you specify an exact domain like you will need to serve a key at a specific domain. It will look something like this.

Create a file containing just this data:
And make it available on your web server at this URL:

If you use a wildcard * you will be required to add a TXT record to your DNS

Please deploy a DNS TXT record under the name with the following value:


Before continuing, verify the record is deployed.

After completing the verification, you will have the certificates in ~/letsencrypt/live

Deploy Concourse to AWS using concourse-up

Now we have certificates and AWS all ready to go so let's deploy Concourse.

Download the latest release of concourse-up from github and put it in your path.

Now in one simple command our concourse deploy will begin

concourse-up \
    deploy \
    --domain \
    --tls-cert "$(cat $HOME/letsencrypt/live/" \
    --tls-key "$(cat $HOME/letsencrypt/live/" \
    --region "us-east-1" \

Now watch the magic happen!

Next Steps

From here you can deploy a pipeline and start to experiment with all the community resources. Or you could set up a pipeline that will automatically refresh the certs with certbot.

Have fun and happy deploying!


Read More

Related Posts


06.30.2021 | Culture | Katy Scott

At Focused Labs, collaboration is key to how we work together; it helps our teams learn from each other, brings us closer and helps us become more efficient...


06.23.2021 | Culture | Austyn

Late-night feedings and diaper changes, the 3-4 month sleep regression, teething, and a growth spurt all mean I'm getting less sleep than...


05.12.2021 | Culture Backend Frontend | Ryan Taylor

Temporarily disrupts "normal" business operations and allow self-organized teams to rapid prototype around their interest areas


04.27.2021 | Culture | Erin Hochstatter

Several years ago, I'd been trying to find an approach to software consulting that made sense for me [...]


01.28.2021 | Backend | Parker Drake

Recently I found myself needing to validate fields in a Spring Boot controller written in Kotlin...


01.22.2021 | Tutorial | Luke Mueller

⌘+⇧+g is the way to go


01.21.2021 | Devops | Katy G

Kube jobs running wild? To delete successful jobs...

additional accent

171 N Aberdeen St
Suite 400
Chicago, IL 60607
(708) 303-8088

[email protected]

© 2021 FocusedLabs, All Rights Reserved.

  • facebook icon
  • twitter icon
  • linkedin icon
  • github icon