Mastering Docker: A Complete Guide to Containerization and Modern DevOps

3.02K 0 0 0 0

📘 Chapter 3: Docker Compose and Multi-Container Apps (Part 1 of 3)

🧠 What is Docker Compose?

Docker Compose is a tool for defining and managing multi-container Docker applications using a single file, typically named docker-compose.yml.

Instead of running multiple docker run commands manually for each service (web app, database, cache, etc.), Compose lets you configure everything in one place and start the entire stack with a single command:

bash

 

docker-compose up


️ When and Why to Use Docker Compose

Use Case

Benefit

Microservices development

Easily run and connect multiple services (API, DB, UI)

CI/CD pipelines

Set up consistent test environments

Local testing

Simulate production-like environments

Isolated project environments

Define stack behavior per project


📂 Structure of a Docker Compose File

The docker-compose.yml file is written in YAML and defines:

  • The services (containers) to run
  • Their networks, volumes, and environment
  • Port mappings, dependencies, and build contexts

🔹 Basic Syntax Structure

yaml

 

version: '3'

services:

  service_name:

    image: image_name

    build:

      context: .

    ports:

      - "host_port:container_port"

    environment:

      - VAR=value

    volumes:

      - ./data:/app/data

    depends_on:

      - another_service


🛠️ Example: Node.js App with MongoDB

Let’s say we have:

  • A Node.js backend
  • A MongoDB database
  • We want both to run together in isolated containers

🔹 Folder Structure

pgsql

 

project/

── docker-compose.yml

── app/

│   ── Dockerfile

│   ── server.js

│   └── package.json


🔹 Dockerfile (inside app/ folder)

Dockerfile

 

FROM node:18

WORKDIR /app

COPY package*.json ./

RUN npm install

COPY . .

CMD ["node", "server.js"]


🔹 docker-compose.yml (root level)

yaml

 

version: '3.8'

 

services:

  web:

    build: ./app

    ports:

      - "3000:3000"

    depends_on:

      - mongo

    environment:

      - MONGO_URL=mongodb://mongo:27017/mydb

 

  mongo:

    image: mongo

    volumes:

      - mongo-data:/data/db

 

volumes:

  mongo-data:


🔍 Breakdown of Services

Service

Purpose

Notes

web

Node.js app container

Built from local Dockerfile in ./app

mongo

MongoDB official image

Uses named volume mongo-data for persistence


️ Running the Stack

In the terminal:

bash

 

docker-compose up

This will:

  • Build the Node.js image
  • Pull the MongoDB image
  • Start both services
  • Set up a private network so web can talk to mongo

You can visit http://localhost:3000 (assuming your app serves on port 3000).


📍 Verifying Service Status

To see running containers:

bash

 

docker-compose ps

To stop and remove all containers:

bash

 

docker-compose down

To rebuild containers:

bash

 

docker-compose up --build

 

🌐 Networking in Docker Compose

By default, Docker Compose creates a single network for all defined services. This enables containers to communicate with each other using service names as hostnames.


🔹 Example: web talks to mongo

In our earlier setup:

yaml

 

environment:

  - MONGO_URL=mongodb://mongo:27017/mydb

The web service can access MongoDB at mongo:27017 (service name = hostname), thanks to Docker’s internal DNS.


🔹 Inspecting the Network

bash

 

docker network ls

docker network inspect project_default

This reveals which containers are connected and how they resolve names.


🔹 Defining Custom Networks (Optional)

yaml

 

networks:

  mynet:

    driver: bridge

 

services:

  web:

    ...

    networks:

      - mynet

  mongo:

    ...

    networks:

      - mynet

🔐 Custom networks allow advanced isolation, shared services, or inter-project linking.


🌱 Using Environment Variables and .env Files

Instead of hardcoding variables in your docker-compose.yml, you can externalize them.


🔹 Creating a .env File

ini

 

PORT=3000

MONGO_HOST=mongo

DB_NAME=mydb


🔹 Reference in Compose File

yaml

 

environment:

  - MONGO_URL=mongodb://${MONGO_HOST}:27017/${DB_NAME}

ports:

  - "${PORT}:${PORT}"

📌 Docker Compose automatically reads .env from the same directory as your YAML file.


💾 Managing Persistent Data with Volumes

Containers are ephemeral. Volumes allow data to persist even after container restarts.


🔹 Named Volumes in Compose

yaml

 

services:

  mongo:

    image: mongo

    volumes:

      - mongo-data:/data/db

 

volumes:

  mongo-data:

🔹 Bind Mounts (Local Folder to Container)

yaml

 

services:

  web:

    volumes:

      - ./app:/app

Volume Type

Purpose

Named

Managed by Docker, ideal for databases

Bind Mount

Maps host directory into container


🔍 View Volumes

bash

 

docker volume ls

docker volume inspect mongo-data


📈 Scaling Services

Docker Compose lets you scale any service that runs the same container image.


🔹 Example: Scale 3 web containers

bash

 

docker-compose up --scale web=3

All web containers will share the same network and load can be distributed (e.g., with Nginx or Traefik as reverse proxy).


🧠 Tips on Scaling

  • Only works for stateless services (e.g., web servers)
  • Use reverse proxy or service discovery for balancing
  • Stateful services like databases should NOT be scaled this way

🚨 Restart Policies

You can make services restart automatically after crashes:

yaml

 

restart: always

Options:

  • no (default)
  • on-failure
  • always
  • unless-stopped

 

🌍 Real-World Example #1: WordPress + MySQL

Let’s run a full WordPress site locally with just Docker Compose!


🔹 docker-compose.yml

yaml

 

version: '3.8'

 

services:

  db:

    image: mysql:5.7

    restart: always

    environment:

      MYSQL_DATABASE: wordpress

      MYSQL_USER: user

      MYSQL_PASSWORD: secret

      MYSQL_ROOT_PASSWORD: rootpass

    volumes:

      - db_data:/var/lib/mysql

 

  wordpress:

    image: wordpress:latest

    restart: always

    ports:

      - "8080:80"

    environment:

      WORDPRESS_DB_HOST: db:3306

      WORDPRESS_DB_USER: user

      WORDPRESS_DB_PASSWORD: secret

      WORDPRESS_DB_NAME: wordpress

    depends_on:

      - db

 

volumes:

  db_data:


️ Start Everything

bash

 

docker-compose up -d

Visit http://localhost:8080 to set up WordPress.


🧪 Real-World Example #2: MEAN Stack (MongoDB + Express + Angular + Node.js)

Using Compose, you can define all services in one file and connect them seamlessly:

  • api → Node.js + Express
  • frontend → Angular
  • db → MongoDB

📌 This setup is ideal for teams collaborating on full-stack projects.


🧰 Common Docker Compose Commands

Command

Purpose

docker-compose up

Start services

docker-compose up -d

Start in detached mode

docker-compose down

Stop and remove services and network

docker-compose build

Build/rebuild services

docker-compose restart

Restart all services

docker-compose logs

View logs from all services

docker-compose exec <svc>

Run commands inside a container


🐛 Troubleshooting Tips

Issue

Fix

Port already in use

Change ports: mapping in YAML

Container restarts repeatedly

Run docker-compose logs to inspect errors

Service fails due to dependency

Use depends_on: or start services with a short delay

Changes not reflecting

Use docker-compose up --build and check volume mount conflicts

Permission denied (volumes)

Ensure correct file permissions and use non-root users


🔒 Best Practices for Compose Projects

Practice

Why It Matters

Use .env files

Decouple config from source code

Separate prod/dev YAMLs

Define overrides in docker-compose.override.yml

Limit volume sharing

Avoid unintentional file overwrites

Clean up with down -v

Remove volumes when resetting environments

Use versioned base images

Avoid latest to ensure consistency

Commit your Compose files

Make environments reproducible for the whole team


📁 Dev vs Prod: Compose Override Strategy

Split environments into multiple files:

bash

 

docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

docker-compose.prod.yml may define different environment variables, optimized images, or restrict volumes.


Summary: Chapter 3 Wrap-Up


Concept

Takeaway

Compose Basics

Orchestrates multiple services with one file

Service Networking

Internal DNS allows container-to-container communication

Environment Management

Use .env and custom YAML files for flexible configuration

Real-World Use Cases

Run entire app stacks (e.g., WordPress, MEAN stack) easily

Debugging and Logs

Use logs, ps, exec to diagnose and fix issues quickly

Back

FAQs


1. Q: What exactly is Docker and how is it different from a virtual machine (VM)?

A: Docker is a containerization platform that allows applications to run in isolated environments. Unlike VMs, Docker containers share the host OS kernel and are much more lightweight and faster to start.

2. Q: What is a Docker container?

A: A Docker container is a runnable instance of a Docker image. It includes everything needed to run an application: code, runtime, libraries, and dependencies—all in an isolated environment.

3. Q: What is the difference between a Docker image and a Docker container?

A: A Docker image is a read-only blueprint or template used to create containers. A container is the live, running instance of that image.

4. Q: What is Docker Hub?

A: Docker Hub is a cloud-based repository where developers can share and access Docker images. It includes both official and community-contributed images.

5. Q: What is a Dockerfile?

A: A Dockerfile is a script that contains a series of commands and instructions used to create a Docker image. It defines what goes into the image, such as the base OS, software dependencies, and run commands.

6. Q: Can Docker run on Windows or macOS?

A: Yes! Docker Desktop is available for both Windows and macOS. It uses a lightweight VM under the hood to run Linux-based containers.

7. Q: How is Docker used in DevOps?

A: Docker streamlines development, testing, and deployment by providing consistent environments. It integrates well with CI/CD pipelines, automates deployments, and simplifies rollback strategies.

8. Q: What is Docker Compose and why use it?

A: Docker Compose is a tool for defining and managing multi-container Docker applications using a YAML file. It's ideal for setting up development environments with multiple services (e.g., web + database).

9. Q: Is Docker secure?

A: Docker offers strong isolation but not complete security out-of-the-box. Best practices like using minimal base images, non-root users, and scanning for vulnerabilities are recommended.

10. Q: What are some real-world use cases of Docker?

A: Docker is used for local development environments, microservices deployment, machine learning pipelines, CI/CD workflows, cloud-native apps, and legacy app modernization.