docker - 在Jenkins服务器中调试Docker构建

Question

我有一个Jenkins服务器，用于构建docker镜像(使用docker maven插件)。

这些构建通常需要大约40秒。但有时它们最多需要1.5个小时。

现在我想知道为什么。我也想知道如何调试这种情况。
jenkins控制台中构建的输出仅告诉我它在maven docker构建期间挂起。挂起的示例输出:

[INFO] ------------------------------------------------------------------------
[INFO] Building MY Docker Image MY Image 0.5.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] >>> docker-maven-plugin:0.15.14:build (default-cli) > initialize @ docker-image-my-image >>>
[INFO] 
[INFO] <<< docker-maven-plugin:0.15.14:build (default-cli) < initialize @ docker-image-my-image <<<
[INFO] 
[INFO] --- docker-maven-plugin:0.15.14:build (default-cli) @ docker-image-my-image ---
[INFO] Building tar: /usr/home/jenkinshome/workspace/JOB_NAME/source-repo/docker/image-my-image/docker-build.tar
[INFO] DOCKER> docker-build.tar: Created [my-image] in 69 milliseconds

现在挂了。

我可以以某种方式查看docker卡在哪个操作上吗？

更新

当直接通过docker build启动时，构建也卡在主机机器上。
这是使用strace执行此操作的输出的结尾:

futex(0xc820028908, FUTEX_WAKE, 1)      = 1
clock_gettime(CLOCK_REALTIME, {1486567620, 329675667}) = 0
clock_gettime(CLOCK_REALTIME, {1486567620, 329756643}) = 0
socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/docker.sock"}, 23) = 0
clock_gettime(CLOCK_REALTIME, {1486567620, 330059768}) = 0
epoll_create1(EPOLL_CLOEXEC)            = 4
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=3892185696, u64=139659048782432}}) = 0
getsockname(3, {sa_family=AF_LOCAL, NULL}, [2]) = 0
getpeername(3, {sa_family=AF_LOCAL, sun_path="/var/run/docker.sock"}, [23]) = 0
futex(0xc820028908, FUTEX_WAKE, 1)      = 1
read(3, 0xc820349000, 4096)             = -1 EAGAIN (Resource temporarily unavailable)
write(3, "POST /v1.24/build?buildargs=%7B%"..., 349) = 349
futex(0xc820028d08, FUTEX_WAKE, 1)      = 1
write(3, "7ff\r\nockerfile\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2054) = 2054
ioctl(0, TIOCGWINSZ, {ws_row=57, ws_col=105, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, TIOCGWINSZ, {ws_row=57, ws_col=105, ws_xpixel=0, ws_ypixel=0}) = 0
) = 481, "Sending build context to Docker "..., 48Sending build context to Docker daemon 2.048 kB
write(1, "\r\n", 2
)                     = 2
write(3, "0\r\n\r\n", 5)                = 5
futex(0xc820028d08, FUTEX_WAKE, 1)      = 1
futex(0x12d3a48, FUTEX_WAIT, 0, NULLStep 1 : FROM jboss-db
 ---> 74a0020a9922
Step 2 : MAINTAINER ***
 ---> Using cache
 ---> 5d38cbd5501b
Step 3 : USER root:root
 ---> Running in 64b14554d8be

而此时它挂了

Answer 1

您在这里有很多选择，您必须确定哪种选择最适合您。您可以:

strace Docker主机上的进程。

在主机上使用sysdig之类的东西，它更易于使用且更详细。

进行printf调试或某种日志记录，表明您的构建对于stdout / stderr和docker log -f <build container>仍然有效，以查看是卡住的构建还是jenkins从属/主进程。

任何更详细的建议都可能需要有关您的设置，挂起的含义以及确切挂起了哪个组件的更多信息。