some basics of docker

14 Jun, 2024

Docker deployments for AI applications

Docker is a tool that allows devs to deploy their apps easily in a container to run on any host OS. The benefit of Docker is that it packages an app with all its dependencies into a container.

Containers are highly efficient in terms of memory usage unlike VMs.

VMs

Provides isolation but has high resource overhead and slow boot times.

Containers

Lightweight as they share OS, can be easily deployed.

This mechanism decouples the app from the environment in which they actually run allowing devs to easily deploy into public cloud, private data centre or even in their own laptop regardless of which OS they use.

Docker daemon is the heart and manages. Similar to the k8s api server.
Docker client is used to interact with the daemon.
Images are the blueprints.
Containers run the actual app.
Docker Hub is the registry. Similar to AWS ECR.

Installation

Install Docker based on the OS you have.

Once you install docker, you can test if it is installed correctly by running:

$ docker run hello-world

Pull

[registry-url]/[namespace]/[image]:[tag]

$ docker pull docker pull pytorch/pytorch:2.3.1-cuda11.8-cudnn8-devel

only downloads those layers that haven't been cached locally

Run

run the image in an interactive mode

$ docker run -it pytorch/pytorch:2.3.1-cuda11.8-cudnn8-devel

exit with CTRL+D

you can ssh into running containers docker exec -ti <container_id> bash

listing all containers

$ docker ps

a useful variant is to use -a

to run with GPUs pass --gpus=all

Volumes

persists data beyond the container lifecycle

add -v HOST_PATH:CONTAINER_PATHto docker run command

For example, map the current dir to /opt/local

$ docker run \
    -v `pwd`:/opt/local \
    -it pytorch/pytorch:2.3.1-cuda11.8-cudnn8-devel

Clean up

$ docker stop

$ docker stop $(docker ps -a -q)

$ docker rm

$ docker rmi

Dockerfile

It's a text file that has all the commands

We start with specifying our base image:

FROM pytorch/pytorch:2.3.1-cuda11.8-cudnn8-devel

We can pass ARG

ARG key=value

for example

ARG CUDA="11.8"

FROM pytorch/pytorch:2.3.1-cuda${CUDA}-cudnn8-devel

# set up a directory
WORKDIR /usr/src/app

We can also use ENV which persists within the docker image like ENV DEBIAN_FRONTEND=noninteractive

installing dependencies

RUN pip install --no-cache-dir -r requirements.txt

next copy files

# copy all the files to the container
COPY . .

expose the port

EXPOSE 5000

Best practices

Use multi-stage builds to create leaner and more secure images.
Order commands properly.
Use smaller base images like python:3.8-slim.
Use COPY (copy local files from the Docker host to the image) over ADD (downloading external files).
Cache python packages using --mount=type=cache,target=/root/.cache/pip.
Run only one process per container.
Use ENTRYPOINT instead of CMD.
Use a .dockerignore file.
Use docker scout to scan the image for vulnerabilities.