java - 在 Web 应用程序中卡住

标签 java spring tomcat freeze lag

我在生产系统上遇到这个问题,我不知道下一步该去哪里:

上下文:在 Tomcat 服务器上使用 REST API、Java、Spring、Hibernate 的网络应用。

  • 随机方法(由调用我的 REST API 的用户触发)卡住大约 15 分钟,然后正常返回。没有错误,没有异常。
  • 不可重现,随时可能发生。
  • 它可能发生在非常简单的方法上:它发生在我调用 Spring JPA Repositoy 方法的单行代码上(因此由 Spring 实现,而不是我自己)。
  • 我有 Javamelody,所以我监控数据库响应时间:它不是数据库,不是 SQL!
  • 我有 Spring 监控切入点 (org.springframework.aop.interceptor.PerformanceMonitorInterceptor),所以我知道它“挂起”在 web 应用程序内部。正如我之前所说,我看不到任何模式。它可以卡在任何地方。

如何进行?

  • 如何确定这是否是基础架构问题?
  • 如何查看挂起的内容?
  • 有趣的是相似的时间:总是大约 933000 毫秒或大约 15 分钟。这听起来像是超时,但是哪一个因为没有错误!是否有任何已知的 15 分钟默认超时?

非常感谢或您的帮助。雷米

2016 年 11 月 25 日编辑

获取连接的时间似乎丢失了:

org.hibernate.engine.internal.StatisticalLoggingSessionEventListener - catalina-exec-31 admin 10.226.17.28 - Session Metrics {
    929967181162 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    929967495527 nanoseconds spent preparing 1 JDBC statements;
    449250 nanoseconds spent executing 1 JDBC statements;
    0 nanoseconds spent executing 0 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)
}

The hanging thread is doing "getConnection":

catalina-exec-31
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
java.net.SocketInputStream.read(SocketInputStream.java:170)
java.net.SocketInputStream.read(SocketInputStream.java:141)
java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
java.io.BufferedInputStream.read(BufferedInputStream.java:345)
org.mariadb.jdbc.internal.packet.read.ReadPacketFetcher.getReusableBuffer(ReadPacketFetcher.java:166)
org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:900)
org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:604)
org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:261)
org.mariadb.jdbc.MariaDbStatement.executeQuery(MariaDbStatement.java:383)
org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
org.apache.commons.dbcp.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:658)
org.apache.commons.dbcp.PoolableConnectionFactory.validateObject(PoolableConnectionFactory.java:635)
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1205)
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:139)
org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:380)
org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:228)
org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:171)
org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:63)
org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:162)
org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:186)
org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:160)
org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1885)
org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1862)
org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1839)
org.hibernate.loader.Loader.doQuery(Loader.java:910)
org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:355)
org.hibernate.loader.Loader.doList(Loader.java:2554)
org.hibernate.loader.Loader.doList(Loader.java:2540)
org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2370)
org.hibernate.loader.Loader.list(Loader.java:2365)
org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:497)
org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:387)
org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:236)
org.hibernate.internal.SessionImpl.list(SessionImpl.java:1264)
org.hibernate.internal.QueryImpl.list(QueryImpl.java:103)
org.hibernate.jpa.internal.QueryImpl.list(QueryImpl.java:573)
org.hibernate.jpa.internal.QueryImpl.getResultList(QueryImpl.java:449)
org.hibernate.jpa.criteria.compile.CriteriaQueryTypeQueryAdapter.getResultList(CriteriaQueryTypeQueryAdapter.java:67)
sun.reflect.GeneratedMethodAccessor159.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.springframework.orm.jpa.SharedEntityManagerCreator$DeferredQueryInvocationHandler.invoke(SharedEntityManagerCreator.java:362)
com.sun.proxy.$Proxy146.getResultList(Unknown Source)
org.springframework.data.jpa.repository.query.JpaQueryExecution$CollectionExecution.doExecute(JpaQueryExecution.java:110)
org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:74)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:97)
org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:88)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:395)
org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:373)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.data.repository.core.support.RepositoryFactorySupport$DefaultMethodInvokingMethodInterceptor.invoke(RepositoryFactorySupport.java:486)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281)
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodIntercceptor.invoke(CrudMethodMetadataPostProcessor.java:122)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
com.sun.proxy.$Proxy94.findByType(Unknown Source)
sun.reflect.GeneratedMethodAccessor171.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
org.springframework.aop.interceptor.PerformanceMonitorInterceptor.invokeUnderTrace(PerformanceMonitorInterceptor.java:62)
org.springframework.aop.interceptor.AbstractTraceInterceptor.invoke(AbstractTraceInterceptor.java:112)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
com.sun.proxy.$Proxy124.findByType(Unknown Source)

最佳答案

dbcp 1.4 的这个配置解决了它。

# Database connection pool setup: see http://commons.apache.org/dbcp/api-1.4/org/apache/commons/dbcp/BasicDataSource.html
dbcp.maxWait=30000
dbcp.maxIdle=10
dbcp.maxActive=50
dbcp.validationQuery=SELECT 1 FROM DUAL
dbcp.testWhileIdle=true
dbcp.testOnReturn=true
dbcp.testOnBorrow=true
dbcp.validationQueryTimeout=2
dbcp.accessToUnderlyingConnectionAllowed=true
dbcp.timeBetweenEvictionRunsMillis=60000
dbcp.numTestsPerEvictionRun=5
dbcp.minEvictableIdleTimeMillis=300000
dbcp.removeAbandonedTimeout=60

关于java - 在 Web 应用程序中卡住,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/40680303/

相关文章:

java - 在java中,当我们使用函数 'FileReader(String fileName)' 时,我们应该将文件 fileName 保存在哪里?

java - 为什么有些框架拒绝放弃对古老 Java 版本的支持?

java - 连接超时 spring boot 应用程序和 mysql

java - 包更改后在tomcat中发布失败

java - 如果未登录,则服务器端重定向

java - Android JSON转换问题

java - 如何返回不分页的实体集合? Spring 启动休息

java - 单元测试 REST 客户端——模拟服务

java - 如何选择合适的 JDK JavaEE Spring 版本

java - 组织.postgresql.util.PSQLException : FATAL: database "postgres>" does not exist