jboss - 由于 org.infinispan.util.concurrent.jdk8backported.LongAdder 中的 `EXCEPTION_ACCESS_VIOLATION` 导致 JVM 崩溃

Question

这种情况发生在 JBoss EAP 6.1 上，对于单个用户来说它工作正常，但是一旦 2 或 3 个并发用户开始与应用程序交互，JVM 就会崩溃。

JVM 因 EXCEPTION_ACCESS_VIOLATION 崩溃；完整跟踪如下:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x024c8266, pid=5136, tid=8736
#
# JRE version: 6.0_45-b06
# Java VM: Java HotSpot(TM) Client VM (20.45-b01 mixed mode windows-x86 )
# Problematic frame:
# J  org.infinispan.util.concurrent.jdk8backported.LongAdder.add(J)V
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x6602dc00):  JavaThread "http-ds-7071bc90200f..corp.in/10.112.70.75:8080-6" daemon [_thread_in_Java, id=8736, stack(0x663a0000,0x663f0000)]

siginfo: ExceptionCode=0xc0000005, reading address 0x8ddb6987

Registers:
EAX=0x00000002, EBX=0xffffffff, ECX=0x00000001, EDX=0x0ddb6988
ESP=0x663ee3c0, EBP=0x663eea58, ESI=0x0ddb60a0, EDI=0xffffffff
EIP=0x024c8266, EFLAGS=0x00010293

Top of Stack: (sp=0x663ee3c0)
0x663ee3c0:   0ddb4830 00000000 663eea58 024d2b44
0x663ee3d0:   0ddb1610 0ddb1610 663eea58 024d7368
0x663ee3e0:   0ddb4830 00000000 0ddb1610 0ddb6be0
0x663ee3f0:   22239910 0ddb1610 2223aff0 22239910
0x663ee400:   0ddb60a0 2223c338 2127c660 24bb6ca8
0x663ee410:   24c1f6e0 0ddb6070 00000002 0ddb6988
0x663ee420:   4ac892b3 0ddb1610 663eea58 024d5bf4
0x663ee430:   00000000 00000000 663eea58 024c5a0c 

Instructions: (pc=0x024c8266)
0x024c8246:   0c 00 00 00 b8 01 00 00 00 8b f7 e9 5b 00 00 00
0x024c8256:   8b bc 24 80 00 00 00 8b 9c 24 84 00 00 00 3b 02
0x024c8266:   f2 0f 10 82 ff ff ff 7f 66 0f 7e c6 66 0f 73 d0
0x024c8276:   20 66 0f 7e c0 8b d6 8b c8 03 d7 13 cb 89 34 24 


Register to memory mapping:

EAX=0x00000002 is an unknown value
EBX=0xffffffff is an unknown value
ECX=0x00000001 is an unknown value
EDX=
[error occurred during error reporting (printing register info), id 0xc0000005]

Stack: [0x663a0000,0x663f0000],  sp=0x663ee3c0,  free space=312k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J  org.infinispan.util.concurrent.jdk8backported.LongAdder.add(J)V
j  org.infinispan.CacheImpl.put(Ljava/lang/Object;Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;JLjava/util/concurrent/TimeUnit;Ljava/util/EnumSet;Ljava/lang/ClassLoader;)Ljava/lang/Object;+24
j  org.infinispan.CacheImpl.put(Ljava/lang/Object;Ljava/lang/Object;JLjava/util/concurrent/TimeUnit;JLjava/util/concurrent/TimeUnit;)Ljava/lang/Object;+12
j  org.infinispan.CacheSupport.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+17
j  org.infinispan.AbstractDelegatingCache.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+6
j  org.infinispan.spring.provider.SpringCache.put(Ljava/lang/Object;Ljava/lang/Object;)V+6
j  com.hmtp.security.server.util.SessionTokenValidator.refreshAccessTime(Ljava/lang/String;)V+25
j  com.hmtp.security.server.util.SessionTokenValidator.checkAndRenewToken([Ljava/lang/String;)Lorg/springframework/security/web/authentication/rememberme/PersistentRememberMeToken;+174
j  com.hmtp.security.server.auth.MultiTenantRememberMeServices.processAutoLoginCookie([Ljava/lang/String;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)Lorg/springframework/security/core/userdetails/UserDetails;+5
j  org.springframework.security.web.authentication.rememberme.AbstractRememberMeServices.autoLogin(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)Lorg/springframework/security/core/Authentication;+64
j  com.hmtp.common.security.server.BrownstoneRememberMeAuthenticationFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+20
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
j  org.springframework.security.web.FilterChainProxy.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+40
j  org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(Ljavax/servlet/Filter;Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+5
j  org.springframework.web.filter.DelegatingFilterProxy.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+71
J  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
J  org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+200
j  org.jboss.as.web.session.ClusteredSessionValve.handleRequest(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;Lorg/jboss/servlet/http/HttpEvent;Z)V+61
j  org.jboss.as.web.session.ClusteredSessionValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+26
j  org.jboss.as.web.session.JvmRouteValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+33
j  org.jboss.as.web.session.LockingValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+43
j  org.jboss.as.web.security.SecurityContextAssociationValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+402
J  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V
j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+42
j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+188
j  org.apache.coyote.http11.Http11Processor.process(Ljava/net/Socket;)Lorg/apache/tomcat/util/net/JIoEndpoint$Handler$SocketState;+349
j  org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Ljava/net/Socket;)Lorg/apache/tomcat/util/net/JIoEndpoint$Handler$SocketState;+65
j  org.apache.tomcat.util.net.JIoEndpoint$Worker.run()V+128
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [jvm.dll+0xfb88b]
V  [jvm.dll+0x18d551]
V  [jvm.dll+0xfba31]
V  [jvm.dll+0xfba8b]
V  [jvm.dll+0xb5e89]
V  [jvm.dll+0x119b74]
V  [jvm.dll+0x14217c]
C  [msvcr71.dll+0x9565]  endthreadex+0xa0
C  [kernel32.dll+0x4ee6c]  BaseThreadInitThunk+0x12
C  [ntdll.dll+0x6399b]  RtlInitializeExceptionChain+0xef
C  [ntdll.dll+0x6396e]  RtlInitializeExceptionChain+0xc2

VM state:synchronizing (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00dd8360] Safepoint_lock - owner thread: 0x01a68c00
[0x00dd83c8] Threads_lock - owner thread: 0x01a68c00

Heap
 def new generation   total 400384K, used 175418K [0x03ae0000, 0x1ed50000, 0x1ed80000)
  eden space 355904K,  47% used [0x03ae0000, 0x0df2c4f0, 0x19670000)
  from space 44480K,  16% used [0x1c1e0000, 0x1c8e2540, 0x1ed50000)
  to   space 44480K,   0% used [0x19670000, 0x19670000, 0x1c1e0000)
 tenured generation   total 889536K, used 111605K [0x1ed80000, 0x55230000, 0x552e0000)
   the space 889536K,  12% used [0x1ed80000, 0x25a7d7f0, 0x25a7d800, 0x55230000)
 compacting perm gen  total 102144K, used 102123K [0x552e0000, 0x5b6a0000, 0x652e0000)
   the space 102144K,  99% used [0x552e0000, 0x5b69ad60, 0x5b69ae00, 0x5b6a0000)
No shared spaces configured.

Code Cache  [0x01ae0000, 0x02cb8000, 0x03ae0000)
 total_blobs=9369 nmethods=9146 adapters=163 free_code_cache=14876096 largest_free_block=384


VM Arguments:
jvm_args: -Dprogram.name=standalone.bat -Xms1303M -Xmx1303M -XX:MaxPermSize=256M -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n -Dorg.jboss.boot.log.file=C:\jboss-eap-6.1\standalone\log\server.log -Dlogging.configuration=file:C:\jboss-eap-6.1\standalone/configuration/logging.properties 
java_command: C:\jboss-eap-6.1\jboss-modules.jar -mp C:\jboss-eap-6.1\modules -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=C:\jboss-eap-6.1
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.6.0_45
PATH=C:\ProgramData\Oracle\Java\javapath;C:\Program Files\Java\jdk1.6.0_45\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\apache-maven-3.2.2\bin;C:\Program Files\TortoiseSVN\bin;D:\apache-ant-1.9.4\bin
USERNAME=neeraj.ar
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 10, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows 7 Build 7601 Service Pack 1

CPU:total 2 (2 cores per cpu, 1 threads per core) family 6 model 23 stepping 10, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1

Memory: 4k page, physical 3338764k(929876k free), swap 15625020k(11901304k free)

vm_info: Java HotSpot(TM) Client VM (20.45-b01) for windows-x86 JRE (1.6.0_45-b06), built on Mar 26 2013 13:40:03 by "java_re" with MS VC++ 7.1 (VS2003)

time: Thu Jul 23 18:41:10 2015
elapsed time: 256 seconds

Answer 1

崩溃发生在指令上的 C1 编译代码中

    f20f1082ffffff7f    movsd xmm0, qword ptr [edx+0x7fffffff]

这表示尚未修补的 volatile 长负载。

我相信这是一个 JVM 错误 JDK-6965570或其重复JDK-7004258 .
它很久以前就在 JDK 6u60 中修复了，但你似乎使用了非常旧版本的 JDK。