How To Automate Jekyll Blog Deployment To Amazon S3

OK, so you’ve chosen Jekyll as a tool to generate your static blog. Great choice! Now, how do you get it on the internet so you can share it with the rest of the world? There are many options out there, including GitHub Pages and Heroku, but in this guide we’ll go with Amazon S3. It’s one of the cheapest and most reliable ways to host static content. By the end of this guide we’ll have the following flow fully automated:

Write a blog post → Push it to GitHub → Build and upload it to S3 → New post available at http://myblog.com/new-post

Guide: Host Jekyll blog on Amazon S3

  1. Install Jekyll
  2. Generate a blog
  3. Setup GitHub account
  4. Upload files to GitHub
  5. Create S3 bucket
  6. Configure Travis CI
  7. Enable S3 access for Travis CI
  8. Test blog publishing with Travis CI
  9. Configure blog domain
  10. Publish new article
  11. Redirect www blog url to non-www (optional)

Install Jekyll

Jekyll has a fantastic documentation. It’s one of the main reasons I chose it over other static site generators. Installation should be pretty straight-forward if you follow this Jekyll installation guide.

Generate a blog

If you are already familiar with Jekyll, you can probably set up a blog from scratch but for the purpose of this guide let’s have Jekyll generate a default blog for us.

# Navigate to a directory where you want your Jekyll blog to
# be created. I keep all my projects in /Users/jarsopisak/repos.
cd /Users/jarospisak/repos

# Generate a new blog
jekyll new myblog.com

# Change into the blog directory
cd myblog.com

# Build the blog and launch it
bundle exec jekyll serve

Now open your browser and go to http://localhost:4000. If everything goes fine you should see your new blog. That was easy!

Setup GitHub account

We are going with GitHub for a hosted Git repository because a) it’s awesome and b) it has out of box integration with Travis CI, a tool which we’ll soon use to automatically upload our blog to S3.

Go ahead and register an account at GitHub (if you don’t have one already). After that create a new public repository. It’s important to keep the repository public to benefit from a free Travis CI integration. I like to give my repositories same name as the domain, so for this site I created a repository called jarospisak.com.

To make working with GitHub easier, I recommend to upload your public SSH key to Github. This will enable pushing and pulling files from the respository without username and password.

Upload files to GitHub

Now that we have a GitHub account and our blog is up and running locally, we want to upload the blog files to the newly created GitHub repository. Run the following commands locally:

# Make sure you are in your Jekyll blog directory
cd /Users/jarospisak/myblog.com

# Initiate a new git repository
git init

# Create a .gitignore file and exclude _site directory
echo "_site/" > .gitignore 

# Stage all files for commit
git add .

# Commit changes
git commit -m "Initial commit"

# Add the remote repository (replace with your own repo url)
git remote add origin git@github.com:finspin/jarospisak.com.git 

# Push all files to GitHub
git push -u origin master

Go to the GitHub and verify that all files, except for the _site directory, have been uploaded. The _site directory is a special directory generated by Jekyll. We don’t want to upload that directory directly to GitHub. Instead, as you’ll soon see, we’ll have Travis CI generate it for us.

Create S3 bucket

If you don’t have an AWS account, go ahead and register one. You can use your existing Amazon shopping account credentials or create a completely new account.

We will use Amazon S3 bucket to serve our static blog. Log in to the AWS console, search for the S3 service and create a new bucket. This is important: the name of the S3 bucket must be the same as the domain name of your blog. So if your blog domain name will be myblog.com, your S3 bucket must have the same name.

After the bucket has been created, click on the bucket name and go to Properties and enable Static website hosting. Select Use this bucket to host a website and insert index.html to the Index document field. The bucket is now ready to host the Jekyll blog files.

Configure Travis CI

All the building blocks are now in place, we just need to connect them all together. This is where Travis CI comes into picture. Sign in with your GitHub credentials at www.travis-ci.org. Go to the profile settings and enable your blog repository.

Now go back to your repository on GitHub, go to Settings - Integrations & services, click on the Add service button and select Travis CI from the list of available services. GitHub and Travis CI are now connected but Travis doesn’t yet know what to do with your GitHub repository. We are going to instruct it to do the following for us:

  • checkout (copy) the code from GitHub repository to the Travis CI server
  • build Jekyll blog
  • upload Jekyll blog files to Amazon S3

We can give instructions to Travis CI via .travis.yml file. It’s a structured text file where we define the above mentioned steps.

# Install Ruby and rvm - Ruby Version Manager
language: ruby
rvm:
- 2.3.3

# Install Jekyll
install:
  - gem install jekyll

# Build Jekyll blog
script:
  - jekyll build

# Upload Jekyll blog files to Amazon S3 bucket
deploy:
  on:
    branch: master
  provider: s3
  bucket: NAME_OF_YOUR_BUCKET 
  access_key_id: $AWS_ACCESS_KEY_ID
  secret_access_key: $AWS_SECRET_ACCESS_KEY
  region: us-west-2
  acl: public_read
  skip_cleanup: true
  local_dir: ./_site

You can learn more about how Travis CI works but I’ll highlight a couple of lines here:

  • replace the NAME_OF_YOUR_BUCKET with your actual S3 bucket name
  • make sure you are using the correct region. You can find the region of your bucket in the AWS S3 console.
  • you’ll have to provide your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. This will give Travis CI access to your AWS account to perform the S3 bucket upload operation. More on this in the next step.

Create the above file in the root directory of your blog (e.g. /Users/jarsopisak/repos/myblog.com/.travis.yml).

Enable S3 access for Travis CI

As described in the previous step, Travis CI will need access to your Amazon S3 bucket. We can grant access by creating a new user which will belong to a group that has access to modify create S3 buckets.

Create IAM group with Amazon S3 access

  1. Log in to your AWS console
  2. Search for the IAM service
  3. Click on Groups - Create New Group
  4. Give it some name, e.g. s3-full-access-group
  5. Select the AmazonS3FullAccess policy and save the group

Create IAM user and add it to the group

  1. In the same IAM service select Users - Add user
  2. Name it e.g. travis-ci and tick the Programmatic access box
  3. On the next screen select the s3-full-access-group that you created before
  4. Click Review and Create user

Store AWS secrets in Travis CI

  1. In IAM service go to Users and select the travis-ci user
  2. Go to Security credentials and click on Create access key
  3. Copy both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY and store it somewhere temporarily (e.g. Notepad)
  4. Login to your Travis CI account and go to Accounts
  5. Find your Jekyll blog repository and click on the settings icon next to it
  6. Under Environment variables create 2 new variables:
    • Name: AWS_ACCESS_KEY_ID, Value: copy the value from Notepad
    • Name: AWS_SECRET_ACCESS_KEY, Value: copy the value from Notepad

Important: Make sure you leave the Display value in build log off! Otherwise your AWS secrets will be visible in the Travis CI log.

Test blog publishing with Travis CI

Everything should be now configured so let’s test it. In your GitHub account go to Settings - Integrations & services, click on the Edit button next to the Travis CI service and press the Test service button. Travis CI will now copy all the files from the GitHub repository to a newly spun up virtual machine, install all the necessary software, build our Jekyll blog and upload it to S3.

If everything went fine and there are no errors in the Travis CI build, you should see your blog at http://BUCKET_NAME.s3.amazonaws.com/index.html. The links won’t yet work properly because of how S3 routing works as long you can see the homepage our setup has worked correctly. To get a fully functioning blog, we’ll need to point a domain to it.

Configure blog domain

In the AWS console find the Route53 service. If you already have a domain registered somewhere else, you’ll have to transfer it to Route53 for your S3 blog to work correctly.

You can register a new domain in the Route53 Dashboard. After successful registration (it make take few minutes), make the further configuration:

  1. Go to Hosted zones and click on the Create Hosted Zone
  2. Enter your domain and click the Create button
  3. Click Create Record Set
  4. Leave the name empty and for the Alias option select Yes
  5. In the Alias target select the S3 bucket
  6. Click Create

After the DNS propagates across the servers (takes anywhere from few minutes to 24 hours), your blog will be now available at the domain name you just created.

Publish new article

Time to write a new post and share it with the world! Here’s your new automated work flow:

  1. Create a new post in the _posts directory
  2. Commit it and push it to GitHub
# Navigate to your blog directory
cd /Users/jarospisak/repos/jarospisak.com

# Check the changed files
git status

# Stage new post
git add PATH_TO_YOUR_POST

# Commit the post
git commit -m "My new post"

# Push to the GitHub repository
git push

Now sit back and watch the magic happen. Travis CI will get notified of a new commit, it will start a new build and your post will be published in a few minutes.

Redirect www blog url to non-www (optional)

You probably want your blog to work on both http://myblog.com and http://www.myblog.com URLs. There are a couple of ways how to achieve this but the easiest thing is to create another S3 bucket called www.myblog.com and redirect it to the non-www bucket (the original bucket that holds the blog files).

Create www S3 bucket

  1. In AWS S3 console create a new bucket with www prefix, e.g. www.myblog.com
  2. In the Properties tab select Static website hosting
  3. Select Redirect requests
  4. For Target bucket or domain insert the bucket name without the www prefix, e.g. myblog.com
  5. For Protocol insert http
  6. Click Save

Create www subdomain in Route53

  1. In the AWS Route53 console go to Hosted zones and select your domain
  2. Click Create record set
  3. Enter www in the name field
  4. Select Yes for the Alias option
  5. In the Alias target select the bucket with the www prefix
  6. Click Create

If you now type www.myblog.com in your browser you’ll be redirected to myblog.com. If you prefer the redirection to work other way around (i.e. from non-www to www), store your Jekyll blog files in the S3 bucket with the www prefix and redirect non-www bucket to the www bucket.