Compose for development stacks

Introduction

Dockerfiles are great to build container images.

But what if we work with a complex stack made of multiple containers?

Eventually, we will want to write some custom scripts and automation to build, run, and connect our containers together.

There is a better way: using Docker Compose.

In this section, you will use Compose to bootstrap a development environment.

What is Docker Compose?

Docker Compose (formerly known as fig) is an external tool.

Unlike the Docker Engine, it is written in Python. It's open source as well.

The general idea of Compose is to enable a very simple, powerful onboarding workflow:

  1. Checkout your code.

  2. Run docker-compose up.

  3. Your app is up and running!

Compose overview

This is how you work with Compose:

  • You describe a set (or stack) of containers in a YAML file called docker-compose.yml.

  • You run docker-compose up.

  • Compose automatically pulls images, builds containers, and starts them.

  • Compose can set up links, volumes, and other Docker options for you.

  • Compose can run the containers in the background, or in the foreground.

  • When containers are running in the foreground, their aggregated output is shown.

Before diving in, let's see a small example of Compose in action.

composeup
Figure 64 : composeup

Checking if Compose is installed

If you are using the official training virtual machines, Compose has been pre-installed.

If you are using Docker for Mac/Windows or the Docker Toolbox, Compose comes with them.

If you are on Linux (desktop or server environment), you will need to install Compose from its release page or with pip install docker-compose.

You can always check that it is installed by running:

$ docker-compose --version

Launching Our First Stack with Compose

First step: clone the source code for the app we will be working on.

$ cd
$ git clone git://github.com/jpetazzo/trainingwheels
...
$ cd trainingwheels

Alternative : https://github.com/jpetazzo/figdemo

Second step: start your app.

$ docker-compose up

Watch Compose build and run your app with the correct parameters, including linking the relevant containers together.

Launching Our First Stack with Compose

Verify that the app is running at https://<yourHostIP>:8000.

composeapp
Figure 65 : composeapp

Stopping the app

When you hit ^C, Compose tries to gracefully terminate all of the containers.

After ten seconds (or if you press ^C again) it will forcibly kill them.

The docker-compose.yml file

Here is the file used in the demo:

version: "2"

services:
  www:
    build: www
    ports:
      - 8000:5000
    user: nobody
    environment:
      DEBUG: 1
    command: python counter.py
    volumes:
      - ./www:/src

  redis:
    image: redis

Compose file structure

A Compose file has multiple sections:

  • version is mandatory. (We should use "2" or later; version 1 is deprecated.)

  • services is mandatory. A service is one or more replicas of the same image running as containers.

  • networks is optional and indicates to which networks containers should be connected.
    (By default, containers will be connected on a private, per-compose-file network.)

  • volumes is optional and can define volumes to be used and/or shared by the containers.

Compose file versions

  • Version 1 is legacy and shouldn't be used.

    (If you see a Compose file without version and services, it's a legacy v1 file.)

  • Version 2 added support for networks and volumes.

  • Version 3 added support for deployment options (scaling, rolling updates, etc).

The Docker documentation has excellent information about the Compose file format if you need to know more about versions.

Containers in docker-compose.yml

Each service in the YAML file must contain either build, or image.

  • build indicates a path containing a Dockerfile.

  • image indicates an image name (local, or on a registry).

  • If both are specified, an image will be built from the build directory and named image.

The other parameters are optional.

They encode the parameters that you would typically add to docker run.

Sometimes they have several minor improvements.

Container parameters

  • command indicates what to run (like CMD in a Dockerfile).

  • ports translates to one (or multiple) -p options to map ports.
    You can specify local ports (i.e. x:y to expose public port x).

  • volumes translates to one (or multiple) -v options.
    You can use relative paths here.

For the full list, check: https://docs.docker.com/compose/compose-file/

Compose commands

We already saw docker-compose up, but another one is docker-compose build.

It will execute docker build for all containers mentioning a build path.

It can also be invoked automatically when starting the application:

docker-compose up --build

Another common option is to start containers in the background:

docker-compose up -d

Check container status

It can be tedious to check the status of your containers with docker ps, especially when running multiple apps at the same time.

Compose makes it easier; with docker-compose ps you will see only the status of the containers of the current stack:

$ docker-compose ps
Name                      Command             State           Ports          
----------------------------------------------------------------------------
trainingwheels_redis_1   /entrypoint.sh red   Up      6379/tcp               
trainingwheels_www_1     python counter.py    Up      0.0.0.0:8000->5000/tcp

Cleaning up (1)

If you have started your application in the background with Compose and want to stop it easily, you can use the kill command:

$ docker-compose kill

Likewise, docker-compose rm will let you remove containers (after confirmation):

$ docker-compose rm
Going to remove trainingwheels_redis_1, trainingwheels_www_1
Are you sure? [yN] y
Removing trainingwheels_redis_1...
Removing trainingwheels_www_1...

Cleaning up (2)

Alternatively, docker-compose down will stop and remove containers.

It will also remove other resources, like networks that were created for the application.

$ docker-compose down
Stopping trainingwheels_www_1 ... done
Stopping trainingwheels_redis_1 ... done
Removing trainingwheels_www_1 ... done
Removing trainingwheels_redis_1 ... done

Use docker-compose down -v to remove everything including volumes.

Special handling of volumes

Compose is smart. If your container uses volumes, when you restart your application, Compose will create a new container, but carefully re-use the volumes it was using previously.

This makes it easy to upgrade a stateful service, by pulling its new image and just restarting your stack with Compose.

Compose project name

  • When you run a Compose command, Compose infers the "project name" of your app.

  • By default, the "project name" is the name of the current directory.

  • For instance, if you are in /home/zelda/src/ocarina, the project name is ocarina.

  • All resources created by Compose are tagged with this project name.

  • The project name also appears as a prefix of the names of the resources.

    E.g. in the previous example, service www will create a container ocarina_www_1.

  • The project name can be overridden with docker-compose -p.

Running two copies of the same app

If you want to run two copies of the same app simultaneously, all you have to do is to make sure that each copy has a different project name.

You can:

  • copy your code in a directory with a different name

  • start each copy with docker-compose -p myprojname up

Each copy will run in a different network, totally isolated from the other.

This is ideal to debug regressions, do side-by-side comparisons, etc.