Docker is a lightweight containerization platform that makes deploying software easy. Docker images are created from instructions listed in Dockerfiles and Docker Compose (formerly Fig) makes it easy to build multi-container Docker applications. This is great because it compartmentalizes all of the different services required by the application. This separation of concerns makes it extremely easy to add or remove services to the app, and it makes updating a specific service a breeze. It also guarantees that your projects will be portable across different machines since all of the dependencies are bundled within each container. This allows the containers to be independent of the linux version, making it easy to onboard new engineers or deploy code to production with the single requirement of having Docker installed.

This tutorial will walk you through the steps required to configure Docker to work with an existing Rails application. Docker will need to be installed on your local machine; download instructions can be found here. The installation will include Docker Engine, Docker Cli client, Docker Compose, and Docker Machine. This tutorial will use version 2 of the Compose File Format which requires Compose 1.8.0 and Docker Engine 1.10.0 or later. If you are unfamiliar with Docker, or you just need a refresher, you can find a great tutorial that will familarize you with the Docker commands here and a Docker Compose tutorial here. If you would like to code along, the repository with the source code prior to adding any Docker related files can be cloned from here. The completed repository with all Docker related files can be found here.

This application will use Rails 5, Postgres, Redis, Sidekiq, and nginx. We will start small and just get the application and database wired up. Then we will add Redis and Sidekiq. Once all of those services are working, we will add nginx to buffer requests and reponses between clients and the rails application.

Step 1: Connecting the App and Database

We will be using Docker Compose to streamline building a multi-container application. To demonstrate what it would be like to do this without Docker Compose, take a look at the code below under the section the hard way.

The Hard Way

In this section we will set up the app and the database using only docker.

# runs a new container built from the postgres image
docker run -d -p 5432:5432 --name db -v postgres:/var/lib/postgresql/data postgres:9.4

Running the command docker ps will show that the container is running.

docker ps

We then need to change our rails database.yml file so that it knows to use the db container running postgres. Notice that the host is set to the name of our database container. This works becuase we will link the app and db containers so they will be in the same bridge network and will therefore be aware of each other.

# config/database.yml

default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  port: 5432
  host: db
  username: postgres

development:
  <<: *default
  database: docker-rails-school_development

test:
  <<: *default
  database: docker-rails-school_test

production:
  <<: *default
  database: docker-rails-school_production

We would then need to build the app image based on our docker file.

docker build -t app .

This references a dockerfile located in the same directory as the command and tags the image with the name app. The Dockerfile includes the following code:

# Dockerfile syntax can be referenced here: https://docs.docker.com/engine/reference/builder/

# references the ruby image that we want to use, ruby version 2.3.1
FROM ruby:2.3.1

# build essential is required to compile debian package and libpq-dev is for postgres
# nodejs is our javascript runtime
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs

# set environmental variable
ENV RAILS_ROOT /rails_app

# create tmp/pids directory in rails app root folder
RUN mkdir -p $RAILS_ROOT/tmp/pids

# set working directory to rails root
WORKDIR $RAILS_ROOT

# copy over startup.sh file
COPY config/startup.sh /opt/startup.sh

# copy over Gemfile
COPY Gemfile Gemfile

# copy over Gemfile.lock
COPY Gemfile.lock Gemfile.lock

# install bundler gem
RUN gem install bundler

# run bundle install before copying over the entire app. This way installed gems are
# cached and you only have to wait for bundle install to run again if Gemfile is changed
RUN bundle install

# copy over rails files to $RAILS_ROOT
COPY . .

# run command startup.sh
CMD [ "/opt/startup.sh" ]

The line in the Dockerfile is a command which references the startup.sh file that we copied over on line 12 of our Dockerfile. The startup.sh file runs a few commands and starts the server.

#!/bin/bash

# tells the bash script to exit whenever anything returns a non-zero return value.
set -e

# will check current database schema version. First time, this will throw an
# error in the server logs because the database will not exist. After the error,
# the database will be created, the schema will be loaded and seeded.
rails db:version || bundle exec rails db:setup

# run migrations
rails db:migrate

# start puma
exec bundle exec rackup -p 3000 -o 0.0.0.0

To start the container that we will call app_container run the following command:

docker run -p 3000:3000 --link db:db --name app_container app

You should see the puma server start in development mode. If we wanted the server to run in the background we could have passed the -d flag in the command above.

Leave the server running and open a new tab in your terminal. We can now run docker ps again and we should see both containers running.

docker ps

At this point if you navigate to localhost:3000 you should see the app homepage.

We have no users on the list yet but that is expected. You should be able to add users to the mailing list now but sending emails will not work since we have not setup the Sidekiq and Redis containers yet.

As I am sure you have noticed, typing and remembering all these flags every time you want to get the containers running can be quite cumbersome and we have only covered a few of the options. Luckily, Docker Compose streamlines this process in an easy to use docker-compose.yml file. The rest of this tutorial will use Docker Compose.

Setting Up the App and Database with Docker Compose

To use Docker Compose you will need to create a file called docker-compose.yml in the root folder of the rails application. To setup the app and database add the following to the docker-compose.yml file:

# docker-compose file reference syntax https://docs.docker.com/compose/compose-file/

# version 2 lets docker know we will be using the 2nd version of the docker
# compose file format
version: "2"

# this is where you will configure each container
services:
  # this is our rails app container
  app:
    # build specifies where to find the Dockerfile.  In this case it is in the
    # same directory as the docker-compose.yml file and it has to be named Dockerfile.
    # To specify a file with a different name you will need to use build with
    # context and dockerfile suboptions.
    build: .
    # sets RAILS_ENV to development. We will come back to this later and improve
    # how we deal with multiple environments
    environment:
      - RAILS_ENV=development
    # call startup.sh script
    command: [ "/opt/startup.sh" ]
    # specify ports for host and container to use
    ports:
      - "3000:3000"
    # docker-compose will automatically look in .env for environment variables.
    env_file: .env
    # links other services and specifies dependencies in service to determine startup order
    links:
      - db

  # this is our database container
  db:
    # specifies what image to use to build the container
    image: postgres:9.4
    # named volume which needs to be specified below in the top value volumes key
    volumes:
      - postgres:/var/lib/postgresql/data

# named volumes need to be specified down here so they are created. In docker
# compose version 1, these were created automatically if they did not already exist.
volumes:
  postgres:

The app service is referencing the same startup.sh file that we saw above. If you have not done so already, in the config directory create the startup.sh file. To be sure the proper permissions are set run the following command:

chmod 775 config/startup.sh

Then add the following code to the startup.sh file (changed slightly from above)

#!/bin/bash

# tells the bash script to exit whenever anything returns a non-zero return value.
set -e

# crude way to wait for database container to be ready
echo "Please wait while we allow time for database container to start..."
sleep 20

# will check current database schema version. First time, this will throw an
# error in the server logs because the database will not exist. After the error,
# the database will be created, the schema will be loaded and seeded.
rails db:version || bundle exec rails db:setup

# run migrations
rails db:migrate

# start puma
exec bundle exec rackup -E $RAILS_ENV -p 3000 -o 0.0.0.0

You will need to change the database.yml file like we did above if you have not already.

The Dockerfile will be exactly like the one we used above except we can remove the command from the last line since this will be specified in our docker-compose.yml file. If you have not done so yet, create a file called Dockerfile in the same directory as the docker-compose.yml file.

# references the ruby image that we want to use, ruby version 2.3.1
FROM ruby:2.3.1

# build essential is required to compile debian package and libpq-dev is for postgres
# node-js is our javascript runtime
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs

# set environmental variable
ENV RAILS_ROOT /rails_app

# create tmp/pids directory in rails app root folder
RUN mkdir -p $RAILS_ROOT/tmp/pids

# set working directory to rails root
WORKDIR $RAILS_ROOT

# copy over startup.sh file
COPY config/startup.sh /opt/startup.sh

# copy over Gemfile
COPY Gemfile Gemfile

# copy over Gemfile.lock
COPY Gemfile.lock Gemfile.lock

# install bundler gem
RUN gem install bundler

# run bundle install before copying over the entire app. This way installed gems are
# cached and you only have to wait for bundle install to run again if Gemfile is changed
RUN bundle install

# copy over rails files to $RAILS_ROOT
COPY . .

At this point I like to create the .env file in the root directory. Before creating the file you should add the .env file to the .gitignore file like so:

# See https://help.github.com/articles/ignoring-files for more about ignoring files.
#
# If you find yourself ignoring temporary files generated by your text editor
# or operating system, you probably want to add a global ignore instead:
#   git config --global core.excludesfile '~/.gitignore_global'

# Ignore bundler config.
/.bundle

# Ignore all logfiles and tempfiles.
/log/*
/tmp/*
!/log/.keep
!/tmp/.keep

# Ignore Byebug command history file.
.byebug_history

.env

You should then commit your changes prior to creating the file. If you have already checked the file in you will need to run the command

git rm --cached FILENAME

This will stop tracking the file. More info on .gitignore can be found here.

Once that is done, you have the option to specify the COMPOSE_PROJECT_NAME. This will be prepended to the names of each of your services as well as the bridge network name that will contain all of the containers. If you choose not to specify a compose project name then the name of the project directory will be prepended to each service. We will specify the docker compose project name in this tutorial so inside of the .env file I would add the following:

COMPOSE_PROJECT_NAME=railsschool

Run the following command to build the containers with docker-compose:

docker-compose build

Then start the containers by running:

docker-compose up -d

Now if you navigate to localhost:3000 you should see the landing page.

At this point we need to create the .dockerignore file. We first need to add it to the .gitignore file like we did above and commit the changes. Next create the .dockerignore file in the root directory of the rails application. We can now add files to this file that we want docker to exclude from our Docker images just like .gitignore does for git.

Next, we need to add docker-compose.override.yml to the .dockerignore file. You should also add .gitignore to the .dockerignore file. Your .dockerignore file should look like this:

docker-compose.override.yml
.gitignore

Commit these changes once this has been done and create the docker-compose.override.yml file in the root directory of the rails app.

You may notice that if you try to make a change to the app right now you have to rebuild the container every time to actually see the changes take affect. This is hardly a development environment so lets fix this by adding the following to the docker-compose.override.yml file:

version: "2"

services:
  app:
    # this file is in our .dockerignore file and overrides the docker-compose.yml
    # file which allows us to set RAILS_ENV to development here then we can set
    # RAILS_ENV to production in the docker-compose.yml file
    environment:
      - RAILS_ENV=development
    volumes:
      # path on host relative to the compose file. In this case we want the entire app.
      - .:/rails_app

You can now change the RAILS_ENV variable in the docker-compose.yml file to equal production like so:

  # we override this in our docker-compose.override.yml file to equal development.
  # Since we included the override file in our .dockerignore and .gitignore files
  # we know that the default will be production when deployed.
  environment:
    - RAILS_ENV=production

Stop and restart the containers by running

docker-compose stop
docker-compose up -d

You should be able to edit files now. This works because we are mounting the entire app as a volume, so the changes are updated in real time. This can be done directly in the docker-compose.yml file but it is nice to put it in the docker-compose.override.yml file because then the containers in production are more fully contained.

The app and database are fully functional at this point so it is time to add the Sidekiq and Redis services.

Step 2: Adding Sidekiq and Redis

To add the sidekiq and redis services to the app, add the following to the docker-compose.yml:

# version 2 lets docker know we will be using the 2nd version of the docker compose file format
version: "2"

# this is where you will configure each container
services:
  # this is our rails app container
  app:
    # build specifies where to find the Dockerfile.  In this case it is in the same
    # directory as the docker-compose.yml file and it has to be named Dockerfile. To
    # specify a file with a different name you will need to use build with context
    # and dockerfile suboptions.
    build: .
    # sets RAILS_ENV to production. This value is overriden in the docker-compose.override.yml file
    environment:
      - RAILS_ENV=production
    # call startup.sh script
    command: [ "/opt/startup.sh" ]
    # specify ports for host and container to use
    ports:
      - "3000:3000"
    # docker-compose will automatically look in .env for environment variables.
    env_file: .env
    # links other services and specifies dependencies in service to determine startup order
    links:
      - db
      - redis

  # this is our database container
  db:
    # specifies what image to use to build the container
    image: postgres:9.4
    # named volume which needs to be specified below in the top value volumes key.
    # naming this volume is optional but I prefer it becuase it is easier for me to
    # remember which volume is associated with which container. By default it will
    # be named a long random string.
    volumes:
      - postgres:/var/lib/postgresql/data

  # redis container
  redis:
    # build container from redis image
    image: redis
    # start the redis server
    command: redis-server
    # port 6379 is the default port redis runs on
    ports:
      - "6379"
    # specify a nmed volume and add it to volumes below so it is created
    volumes:
      - redis:/var/lib/redis/data

  # this is the sidekiq container
  sidekiq:
    # we can use the same dockerfile to build the sidekiq service since we will
    # use the same files
    build: .
    # link sidekiq to both the database and redis
    links:
      - db
      - redis
    # run the sidekiq.yml file
    command: bundle exec sidekiq -C config/sidekiq.yml
    env_file: .env

# named volumes need to be specified down here so they are created. In docker
# compose version 1, these were created automatically if they did not already exist.
volumes:
  postgres:
  redis:

You will also need to add the redis service under the links option for the app service.

Above you will notice that everything required to setup the Sidekiq and Redis services are very similar to what we did for the app and the database. We do however need to add a few things to the application to make Sidekiq work with redis. The first file we need to add is the sidekiq.yml file. Place that in the config directory and add the following code:

# config/sidekiq.yml
# settings should be adjusted accordingly for your application needs

development:
  concurrency: 2
production:
  concurrency: 2
queues:
  - default
  - mailers

Next we need to add the sidekiq initializer and point sidekiq at the redis url. Inside of config/initializers/ create a sidekiq.rb file and add the following lines of code:

# config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: ENV["REDIS_URL"] }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV["REDIS_URL"] }
end

Lastly create the REDIS_URL environmental variable:

REDIS_URL=redis://redis:6379/

After this configuration, the app should be able to send emails and the last thing that is needed is to configure action mailer. This has already been done for you in the config/environments/development.rb file. If you open the file, you should see the following:

# config/environments/development.rb

config.action_mailer.perform_caching = false
config.action_mailer.perform_deliveries = true
config.action_mailer.raise_delivery_errors = true
config.action_mailer.default_url_options = { host: "localhost:3000", port: 3000}
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
  address: "smtp.gmail.com",
  port: 587,
  domain: "my_app.com",
  authentication: "plain",
  enable_starttls_auto: true,
  user_name: ENV['GMAIL_USER_NAME'],
  password: ENV['GMAIL_PASSWORD']
}

You will need to add the following environmental variables:

GMAIL_USER_NAME=YOUR_USER_NAME
GMAIL_PASSWORD=YOUR_GMAIL_PASSWORD

Modifying them to reflect your username and password. If you do not want to use gmail, you can configure action mailer accordingly.

Now that all of this is done, you can run

docker-compose stop
docker-compose up

This will restart all the services and open the server logs in the same terminal window. I prefer this because I can then look at the logs as things are happening. If you prefer you can pass the -d flag to the docker-compose up command to run everything in daemon mode.

At this point the app should be functioning on your local machine as it would if you were not using docker! Our final step will be adding nginx which is recommended in production for scalability for a multitenancy scenario.

Step 3: Adding NGINX

We will be using nginx as a reverse proxy to load balance our application. To get started lets add the nginx service to our docker-compose.yml file. I am going to call this service nginx.

# version 2 lets docker know we will be using the 2nd version of the docker compose file format
version: "2"

# this is where you will configure each container
services:
  # this is our rails app container
  app:
    # build specifies where to find the Dockerfile.  In this case it is in the
    # same directory as the docker-compose.yml file and it has to be named Dockerfile.
    # to specify a file with a different name you will need to use build with context
    # and dockerfile suboptions.
    build: .
    # sets RAILS_ENV to production. This value is overriden in the docker-compose.override.yml file
    environment:
      - RAILS_ENV=production
    # call startup.sh script
    command: [ "/opt/startup.sh" ]
    ports:
      - "3000:3000"
    # docker-compose will automatically look in .env for environment variables.
    env_file: .env
    # links other services and specifies dependencies in service to determine startup order
    links:
      - db
      - redis

  # this is our database container
  db:
    # specifies what image to use to build the container
    image: postgres:9.4
    # named volume which needs to be specified below in the top value volumes key
    volumes:
      - postgres:/var/lib/postgresql/data

  # redis container
  redis:
    # build container from redis image
    image: redis
    # start the redis server
    command: redis-server
    # port 6379 is the default port redis runs on
    ports:
      - "6379"
    # specify a nmed volume and add it to volumes below so it is created
    volumes:
      - redis:/var/lib/redis/data

  # this is the sidekiq container
  sidekiq:
    # we can use the same dockerfile to build the sidekiq service since we will
    # use the same files
    build: .
    # link sidekiq to both the database and redis
    links:
      - db
      - redis
    # run the sidekiq.yml file
    command: bundle exec sidekiq -C config/sidekiq.yml
    env_file: .env

  # nginx container
  nginx:
    # this time we need to tell docker-compose where to find the docker file and
    # what it is called so we use build with context and dockerfile nested options
    build:
      context: .
      dockerfile: config/nginx-Dockerfile
    # link it to the app
    links:
      - app
    # we want to use port 80 which is the default port to listen to webclients on.
    ports:
      - "80:80"


# named volumes need to be specified down here so they are created. In docker
# compose version 1, these were created automatically if they did not already exist.
volumes:
  postgres:
  redis:

Next we will need to create and add the nginx dockerfile. Create a file called nginx-Dockerfile and add it the config directory. Once that is done add the following code to the nginx-Dockerfile.

# config/nginx-Dockerfile

# use the nginx image
FROM nginx

# set environment variable
ENV RAILS_ROOT /rails_app

# set work directory
WORKDIR $RAILS_ROOT

# create a log directory where we will place the nginx log files
RUN mkdir log

# copy over public static files as nginx can serve this more quickly than our app
COPY public public/

# copy of the nginx configuration file to our container
COPY config/nginx.conf /etc/nginx/nginx.conf

# start nginx
CMD [ "/usr/sbin/nginx" ]

Now we will need to create the nginx.conf file. Add the nginx.conf file to the config directory and add the following code:

# config/nginx.conf
# for detailed nginx info reference the nginx docs at https://nginx.org/en/docs/

daemon off;
worker_processes 1;
pid /var/run/nginx.pid;

events {
  worker_connections 1024;
}

http {
  include mime.types;
  default_type application/octet-stream;

  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;

  sendfile on;

  tcp_nopush on;
  tcp_nodelay off;

  gzip on;
  gzip_http_version 1.0;
  gzip_proxied any;
  gzip_min_length 500;
  gzip_disable "MSIE [1-6]\.";
  gzip_types text/plain text/xml text/css
             text/comma-separated-values
             text/javascript application/x-javascript
             application/atom+xml;

  upstream rails_app {
    server app:3000;
  }

  root /rails_app/public;

  server {
    server_name rails_school;

    try_files $uri/index.html $uri.html $uri @rails;

    location @rails {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

      proxy_set_header Host $http_host;

      proxy_redirect off;

      proxy_pass http://rails_app;
    }
  }
}

Finally lets add a volume to the web service in our docker-compose.override.yml file.

version: "2"

services:
  app:
    environment:
      - RAILS_ENV=development
    volumes:
      - .:/rails_app

  nginx:
    volumes_from:
      - app

Now we will just need to make a few changes to our docker-compose.yml file. We want to remove the ports options from the web service and instead expose port 3000. This is more secure since it exposes the port to other linked services but not to the host machine. Your final docker-compose.yml file should look like this.

# version 2 lets docker know we will be using the 2nd version of the docker compose file format
version: "2"

# this is where you will configure each container
services:
  # this is our rails app container
  app:
    # build specifies where to find the Dockerfile.  In this case it is in the same
    # directory as the docker-compose.yml file and it has to be named Dockerfile.
    # To specify a file with a different name you will need to use build with context
    # and dockerfile suboptions.
    build: .
    # sets RAILS_ENV to production. This value is overriden in the docker-compose.override.yml file
    environment:
      - RAILS_ENV=production
    # call startup.sh script
    command: [ "/opt/startup.sh" ]
    expose:
      - "3000"
    # docker-compose will automatically look in .env for environment variables.
    env_file: .env
    # links other services and specifies dependencies in service to determine startup order
    links:
      - db
      - redis

  # this is our database container
  db:
    # specifies what image to use to build the container
    image: postgres:9.4
    # named volume which needs to be specified below in the top value volumes key
    volumes:
      - postgres:/var/lib/postgresql/data

  # redis container
  redis:
    # build container from redis image
    image: redis
    # start the redis server
    command: redis-server
    # port 6379 is the default port redis runs on
    ports:
      - "6379"
    # specify a nmed volume and add it to volumes below so it is created
    volumes:
      - redis:/var/lib/redis/data

  # this is the sidekiq container
  sidekiq:
    # we can use the same dockerfile to build the sidekiq service since we will
    # use the same files
    build: .
    # link sidekiq to both the database and redis
    links:
      - db
      - redis
    # run the sidekiq.yml file
    command: bundle exec sidekiq -C config/sidekiq.yml
    env_file: .env

  # nginx container
  nginx:
    # this time we need to tell docker-compose where to find the docker file and
    # what it is called so we use build with context and dockerfile nested options
    build:
      context: .
      dockerfile: config/nginx-Dockerfile
    # link it to the app
    links:
      - app
    # we want to use port 80 which is the default port to listen to webclients on.
    ports:
      - "80:80"


# named volumes need to be specified down here so they are created. In docker
# compose version 1, these were created automatically if they did not already exist.
volumes:
  postgres:
  redis:

We should be able to stop our previous running containers and restart them with nginx now.

docker-compose stop
docker-compose up

Now you should be able to navigate to localhost:80 and see the application running.

Next Steps

The development environment for this rails application has now been setup and should mirror developing on your local machine pretty closely. In the next post we will discuss deploying the application to production and setting up continuous integration.