Deploying Concourse with concourse-up
01.01.2020 | Testing Devops Backend | Austin Vance
There are a lot of Continuous integration tools out there, and they all have minor pros and cons but are generally the same. You push some code, that code triggers a build, and that build could trigger others. Sometimes these are called pipelines, but the only actual implementation of a pipeline build system I have seen is Concourse. There will be future posts on what a pipeline is and why it's such a powerful concept not that's not this post.
For this post, you should be up and running with Concourse in AWS with SSL and have your first pipeline in about 45 minutes, and most of that time will be waiting for AWS.
With this guide, you will
- Prepare AWS
- Provision valid SSL Certificates for free
- Deploy Concourse to AWS using concourse-up
You will need:
- An internet connection
- An AWS Account with some credit
- A Domain with access to the DNS
You don't want to give concourse-up too many permissions so let's create some restricted keys and use those for our deployment.
First login to AWS and head to the
If you're not familiar with AWS'
IAM rules I recommend reading up on them. There are ton's of guides out there that can help.
I like to use Groups to manage Permissions so first create a group by going to the Group's section and then
Add New Group. After naming your group, you will be prompted to attach security policies.
Add the following:
You should see your new group with 0 users in the group list.
Now we need to add a user. In the
Users section select
Add User, name the user, and check the box
Programmatic access. On the next section add the User to the group you created above. You don't need any tags so finish building the user.
This next screen is essential When you see Success click
Download CSV and put it in a secure place.
Now you're ready to get your SSL Certs ready
Provision valid SSL Certificates
Concourse uses SSL for all of it's service to service communication. You also want to make sure that any content you serve is also SSL so let's do that next.
Certbot has a ton of options and can be really easy depending on your registrar and where you manage your DNS.
The most basic way is to use Certbot's manual image to generate your certificates.
sudo docker run -it --rm --name certbot \ -v "$HOME/letsencrypt:/etc/letsencrypt" \ -v "$HOME/letsencrypt:/var/lib/letsencrypt" \ certbot/certbot certonly \ --manual
Follow the prompts. You will be required to verify ownership of your domain. If you specify an exact domain like ci.focusedlabs.io you will need to serve a key at a specific domain. It will look something like this.
Create a file containing just this data: 6z1rbMF-R6XdSo-s8HrTMKEeLGIvszkIA1IJPqzr9iU.FUHXZSFi6r53NiLV9xzaHFovW4oXb9fjR0j9KNF36tw And make it available on your web server at this URL: http://ci.focusedlabs.io/.well-known/acme-challenge
If you use a wildcard
*.focusedlabs.io you will be required to add a TXT record to your DNS
Please deploy a DNS TXT record under the name _acme-challenge.focusedlabs.io with the following value: 6gcFZfRzUVujnYkhMEF8po1i55e4NscuSFnUcnYR4CQ Before continuing, verify the record is deployed.
After completing the verification, you will have the certificates in
Deploy Concourse to AWS using concourse-up
Now we have certificates and AWS all ready to go so let's deploy Concourse.
Download the latest release of
concourse-up from github https://github.com/EngineerBetter/concourse-up/releases and put it in your path.
Now in one simple command our concourse deploy will begin
concourse-up \ deploy \ --domain ci.focusedlabs.io \ --tls-cert "$(cat $HOME/letsencrypt/live/focusedlabs.io/fullchain.pem)" \ --tls-key "$(cat $HOME/letsencrypt/live/focusedlabs.io/privkey.pem)" \ --region "us-east-1" \ focusedci
Now watch the magic happen!
From here you can deploy a pipeline and start to experiment with all the community resources. Or you could set up a pipeline that will automatically refresh the certs with certbot.
Have fun and happy deploying!
06.30.2021 | Culture | Katy Scott
At Focused Labs, collaboration is key to how we work together; it helps our teams learn from each other, brings us closer and helps us become more efficient...
06.23.2021 | Culture | Austyn
Late-night feedings and diaper changes, the 3-4 month sleep regression, teething, and a growth spurt all mean I'm getting less sleep than...
05.12.2021 | Culture Backend Frontend | Ryan Taylor
Temporarily disrupts "normal" business operations and allow self-organized teams to rapid prototype around their interest areas
04.27.2021 | Culture | Erin Hochstatter
Several years ago, I'd been trying to find an approach to software consulting that made sense for me [...]
01.28.2021 | Backend | Parker Drake
Recently I found myself needing to validate fields in a Spring Boot controller written in Kotlin...
01.22.2021 | Tutorial | Luke Mueller
⌘+⇧+g is the way to go
01.21.2021 | Devops | Katy G
Kube jobs running wild? To delete successful jobs...