docker - 了解多阶段 Dockerfile 的工作流程

Question

当涉及到多阶段时，我正在努力思考一些流程 Dockerfile .

以此为例，我下面有几个问题:

# Dockerfile
# Uses multi-stage builds requiring Docker 17.05 or higher
# See https://docs.docker.com/develop/develop-images/multistage-build/

# Creating a python base with shared environment variables
FROM python:3.8.1-slim as python-base
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    POETRY_HOME="/opt/poetry" \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    POETRY_NO_INTERACTION=1 \
    PYSETUP_PATH="/opt/pysetup" \
    VENV_PATH="/opt/pysetup/.venv"

ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"


# builder-base is used to build dependencies
FROM python-base as builder-base
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        curl \
        build-essential

# Install Poetry - respects $POETRY_VERSION & $POETRY_HOME
ENV POETRY_VERSION=1.0.5
RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

# We copy our Python requirements here to cache them
# and install only runtime deps using poetry
WORKDIR $PYSETUP_PATH
COPY ./poetry.lock ./pyproject.toml ./
RUN poetry install --no-dev  # respects 


# 'development' stage installs all dev deps and can be used to develop code.
# For example using docker-compose to mount local volume under /app
FROM python-base as development
ENV FASTAPI_ENV=development

# Copying poetry and venv into image
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

# Copying in our entrypoint
COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

# venv already has runtime deps installed we get a quicker install
WORKDIR $PYSETUP_PATH
RUN poetry install

WORKDIR /app
COPY . .

EXPOSE 8000
ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD ["uvicorn", "--reload", "--host=0.0.0.0", "--port=8000", "main:app"]


# 'lint' stage runs black and isort
# running in check mode means build will fail if any linting errors occur
FROM development AS lint
RUN black --config ./pyproject.toml --check app tests
RUN isort --settings-path ./pyproject.toml --recursive --check-only
CMD ["tail", "-f", "/dev/null"]


# 'test' stage runs our unit tests with pytest and
# coverage.  Build will fail if test coverage is under 95%
FROM development AS test
RUN coverage run --rcfile ./pyproject.toml -m pytest ./tests
RUN coverage report --fail-under 95


# 'production' stage uses the clean 'python-base' stage and copyies
# in only our runtime deps that were installed in the 'builder-base'
FROM python-base as production
ENV FASTAPI_ENV=production

COPY --from=builder-base $VENV_PATH $VENV_PATH
COPY ./docker/gunicorn_conf.py /gunicorn_conf.py

COPY ./docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

COPY ./app /app
WORKDIR /app

ENTRYPOINT /docker-entrypoint.sh $0 $@
CMD [ "gunicorn", "--worker-class uvicorn.workers.UvicornWorker", "--config /gunicorn_conf.py", "main:app"]

我的问题:

1. 你是docker build ...整个图像，然后只是 docker run ... --target=<stage>运行特定阶段( development 、 test 、 lint 、 production 等)，还是仅构建和运行您需要的特定阶段(例如 docker build ... -t test --target=test && docker run test ... )？

   我想说这不是前者，因为你最终会得到一个带有构建套件的臃肿图像，什么不......正确？

2. 当谈到本地 Kubernetes 开发( minikube 、 skaffold 、 devspace 等)和运行单元测试时，您是否应该引用 Dockerfile 中的这些阶段？ ( devspace 钩子(Hook)或其他东西)或在容器中使用 native 测试工具(例如 npm test 、 ./manage.py test 等)？

感谢您解答这个问题。

Answer 1

从较少的 DevSpace-y 视角和更一般的 Docker-y 视角来回答(没有对 Lukas 的不尊重!):

问题 1

分割

❌ Are you docker build ... this entire image and then just docker run ... --target= to run a specific stage

您的理解很接近，并设法在查询的第二部分中概述了该方法:

✅ or are you only building and running the specific stages you need (e.g. docker build ... -t test --target=test && docker run test ...)?

docker run 命令中不存在 --target 选项，在调用 docker run --help 时可以看到该选项。

I want to say it isn't the former because you end up with a bloated image with build kits and what not... correct?

是的，第一种方法是不可能的，因为当未指定 --target 时，只有最后阶段会合并到您的图像中。这是一个很大的好处，因为它减少了容器的最终大小，同时允许您使用多个指令。

详细信息和示例

这是一个您可以在构建时传入的标志，以便您可以选择专门构建哪些层。这是一个非常有用的指令，可以通过多种不同的方式使用。有一篇不错的博客文章here谈论多阶段构建带来的新功能(--target 就是其中之一)

例如，我已经利用不同的阶段和目标在 CI 中成功构建了相当多的项目，以下是伪代码，但希望上下文能够应用

# Dockerfile
FROM python as base

FROM base as dependencies

COPY requirements.txt .
RUN pip install -r requirements.txt

FROM dependencies as test

COPY src/ src/
COPY test/ test/

FROM dependencies as publish

COPY src/ src/

CMD ...

像这样的 Dockerfile 将使您能够在 CI 工作流程中执行类似的操作，再次是伪代码式的

docker build . -t my-app:unit-test --target test
docker run my-app:unit-test pyunit ...
docker build . -t my-app:latest
docker push ...

在某些情况下，对何时构建的内容进行细粒度控制可能非常有利，并且能够运行那些仅包含几个阶段的图像而无需构建整个应用程序，这是相当有好处的。

这里的关键是，您不需要使用 --target，但它可以用于解决特定问题。

问题 2

When it comes to local Kubernetes development (minikube, skaffold, devspace, etc.) and running unit tests, are you supposed referring to these stages in the Dockerfile (devspace Hooks or something) or using native test tools in the container (e.g. npm test, ./manage.py test, etc.)?

Lukas 很好地涵盖了开发空间特定的方法，但最终您可以根据自己的喜好进行测试。使用 devspace 来更轻松地运行(并记住运行)测试听起来确实是个好主意。无论您使用什么工具来实现更简单的工作流程，很可能仍然会在底层使用 npm test 等。

如果您希望在容器外部调用 npm test ，那没问题，如果您想在容器中调用它，那也没问题。问题的解决方案总是会根据您的情况而变化。 CICD 有助于标准化外部因素并提供统一的方法来确保测试的执行和部署的可审核性

希望对形状或形式有所帮助👍