java - 在停止并重新启动我的应用程序时，我收到 JMX 地址已在使用中的错误

Question

我正在编写一个脚本来停止和重新启动我的服务。理想情况下，如果出现内存不足错误，它将由 gradle.build 文件中设置的 JVM 选项调用。到目前为止，它已被调用并成功终止了该进程，但是当它调用启动脚本时，我收到一条错误，指出我在构建文件中设置的 JMX 端口已在使用中:

Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 9010; nested exception is:
        java.net.BindException: Address already in use (Bind failed)
sun.management.AgentConfigurationError: java.rmi.server.ExportException: Port already in use: 9010; nested exception is:
        java.net.BindException: Address already in use (Bind failed)
        at sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:480)
        at sun.management.Agent.startAgent(Agent.java:262)
        at sun.management.Agent.startAgent(Agent.java:452)
Caused by: java.rmi.server.ExportException: Port already in use: 9010; nested exception is:
        java.net.BindException: Address already in use (Bind failed)
        at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:346)
        at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:254)
        at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
        at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
        at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:236)
        at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:213)
        at sun.rmi.registry.RegistryImpl.<init>(RegistryImpl.java:173)
        at sun.management.jmxremote.SingleEntryRegistry.<init>(SingleEntryRegistry.java:49)
        at sun.management.jmxremote.ConnectorBootstrap.exportMBeanServer(ConnectorBootstrap.java:816)
        at sun.management.jmxremote.ConnectorBootstrap.startRemoteConnectorServer(ConnectorBootstrap.java:468)
        ... 2 more
Caused by: java.net.BindException: Address already in use (Bind failed)

这对我来说没有任何意义，因为我认为终止服务也应该释放端口。

这是我的 JVM 选项:

task appStartScripts(type: CreateStartScripts) {
    def tplName = 'startTemplate.sh'
    assert project.file(tplName).exists()
    defaultJvmOpts = ["-XX:+HeapDumpOnOutOfMemoryError",
                      "-XX:HeapDumpPath=\$HOME/log/",
                      "-Dcom.sun.management.jmxremote",
                      "-Dcom.sun.management.jmxremote.port=9010",
                      "-Dcom.sun.management.jmxremote.authenticate=false",
                      "-Dcom.sun.management.jmxremote.ssl=false",
                      "-Djava.rmi.server.hostname=testHost",
                      "-XX:OnOutOfMemoryError=./restart.sh",
                      "-Xms64m", "-Xmx124m"]
    dependsOn shadowJar
    applicationName = 'start'
    defaultJvmOpts += ["-Dspring.profiles.active=development"]
    classpath = startShadowScripts.classpath
    mainClassName = startShadowScripts.mainClassName
    outputDir = new File(project.buildDir, 'scriptsShadow')

    doLast {
        // IMPORTANT! needed to ensure HOME environment variable is expanded correctly
        unixScript.text = unixScript.text.replace('\\$HOME', '\'"$HOME"\'')
}

这是我的重启脚本:

#!/usr/bin/env sh

envDeploy=`whoami`

if [ "$envDeploy" == "dev_account" ]; then
    envName=development
elif [ "$envDeploy" == "dev_account" ]
then
    envName=quality
elif [ "$envDeploy" == "prod_account" ]
then
    envName=production
fi

PID=`pgrep -f application.jar`

kill -9 $PID
sleep 15
echo "Restarting application for \$envName environment"
./start.sh $envName;

理论上，该服务应该在调用 start.sh 之前完全终止，所以我对我还能做些什么来解决这个问题有点困惑。

编辑: 我还应该提到，重新启动脚本能够在自行调用时很好地终止并重新启动服务:

./restart.sh #typing this into the console works as expected

仅当在内存不足异常时调用此行为时，即调用此行时，才会观察到此行为:

-XX:OnOutOfMemoryError=./restart.sh

编辑2: 我对重新启动脚本进行了一些修改，以尝试查找失败的位置:

#!/usr/bin/env sh

envDeploy=`whoami`

if [ "$envDeploy" == "dev_account" ]; then
    envName=development
elif [ "$envDeploy" == "dev_account" ]
then
    envName=quality
elif [ "$envDeploy" == "prod_account" ]
then
    envName=production
fi

PID=`pgrep -f application.jar` >> check.log

echo "PID is $PID" >> check.log

netstat -nlp | grep $PID >> check.log
kill -9 $PID >> check.log
ps $PID >> check.log
netstat -nlp | grep $PID >> check.log
kill -9 $PID >> check.log
echo "checking defunct processes" >> check.log
ps -ef | grep defunct >> check.log
sleep 15
echo "===========" >> check.log
echo "checking jmx port" >> check.log
netstat -nlp | grep $PID >> check.log
netstat -nlp | grep 9010 >> check.log
kill $PID >> check.log
netstat -tupln |grep ":<Your_Port_Here>" >> check.log
sleep 5 >> check.log
jps >> check.log
echo "Restarting application for $envName environment" >> check.log
./start.sh $envName; >> check.log

添加netstat -nlp | 后grep 9010 >> check.log，我能够从 check.log 看到以下输出:

PID is 11959
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      11959/java
tcp        0      0 0.0.0.0:8081            0.0.0.0:*               LISTEN      11959/java
tcp        0      0 0.0.0.0:9010            0.0.0.0:*               LISTEN      11959/java
tcp        0      0 0.0.0.0:38933           0.0.0.0:*               LISTEN      11959/java
tcp        0      0 0.0.0.0:36257           0.0.0.0:*               LISTEN      11959/java
  PID TTY      STAT   TIME COMMAND
11959 pts/0    Zl     0:38 [java] <defunct>
checking defunct processes
sysibrt+ 13296 13287  0 17:43 pts/0    00:00:00 grep defunct
===========
checking jmx port
tcp        0      0 0.0.0.0:9010            0.0.0.0:*               LISTEN      13287/sh
13370 Jps
Restarting automation-cfg-svc for qa environment

OOm 错误后，端口 9010 的进程 ID 发生变化，表明服务已重新启动。这种情况不应该发生，因为 ./start.sh $envName; 之前没有任何内容来重新启动服务。

Answer 1

只是应用程序没有被脚本杀死，调试并修复这个问题。