grails - Grails堆空间间歇性溢出

标签 grails

我遇到的那些欢乐的问题之一是断断续续,难以复制的。

经过几分钟或几天的运行后,我在测试服务器上的grails应用失败了。我无法在Dev或本地复制。

这里的stacktrace模式示例从中间截去了它的1000行(这将一直这样转储,直到我重新启动Tomcat为止):

at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
Caused by: javax.servlet.ServletException: javax.servlet.ServletException: org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Java heap space
    at net.sf.ehcache.constructs.web.filter.Filter.logThrowable(Filter.java:143)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:91)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
......
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
Caused by: javax.servlet.ServletException: org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Java heap space
    at grails.plugin.cache.web.filter.AbstractFilter.logThrowable(AbstractFilter.java:116)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:70)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)

..................
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at grails.plugin.springcache.web.GrailsFragmentCachingFilter.doFilter(GrailsFragmentCachingFilter.groovy:66)
    at net.sf.ehcache.constructs.web.filter.Filter.doFilter(Filter.java:86)
    at net.bull.javamelody.JspWrapper.invoke(JspWrapper.java:117)
    at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:231)
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
Caused by: org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Java heap space
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    ... 132 more
Caused by: java.lang.OutOfMemoryError: Java heap space

看来可能发生了应用程序错误,调用errorHandler并不断调用自身... kaboom!

环境:
  • Grails:2.0.4
  • Tomcat:apache-tomcat-7.0.28
  • 操作系统:Red Hat Enterprise Linux Server 5.8版(Tikanga)

  • Java :
    java version "1.6.0_22"
    OpenJDK Runtime Environment (IcedTea6 1.10.8) (rhel-1.27.1.10.8.el5_8-x86_64)
    OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
    

    在开发Java中(这可能是我试图让SA在测试系统中更改的主要问题)
    java version "1.6.0_30"
    Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
    Java HotSpot(TM) 64-Bit Server VM (build 20.5-b03, mixed mode)
    

    该服务器当前只有0-5个用户登录,但是此过程在24小时内会消退。

    分配的堆空间为5g,通常使用<20%。

    在Test中发生了另外2件事,而在Dev中没有发生:
  • 日志,最初几乎没有日志记录,但是由于问题,已将其打开,直到DEBUG和RollingFile 100Meg文件在第50个滚动。日志很好,可以记录应用程序特定的内容,直到堆栈崩溃,然后其中唯一的内容就是堆栈,tomcat日志记录也是如此。即使在关闭第50条日志之前就关闭服务器也是如此。因此,我不知道是否发生了某些特定于应用程序的不祥之兆,从而触发了这种情况。
  • JavaMelody插件:图上的标签有点像我3岁的 child 写的那样不可读。我相信所有OS字体都已加载,但可以指向Java版本。

  • 计划程序:
    是的,有2个正在运行,已经运行它们没有问题

    数据库配置:
    遗留数据库有8个数据源,例如,每个当前可能都设置得有点饿:
    def connectionPropertiesMedium = [
            maxActive: 100,
            maxIdle: 30,
            minIdle: 5,
            initialSize: 30,
            testOnBorrow: true,
            testWhileIdle: false,
            testOnReturn: false,
            validationQuery: "SELECT 1",
            minEvictableIdleTimeMillis: 600000,
            timeBetweenEvictionRunsMillis: 600000,
            numTestsPerEvictionRun: 3,
            maxWait: 10000,
            defaultTransactionIsolation: java.sql.Connection.TRANSACTION_READ_UNCOMMITTED
    ]
    

    HeapDump:
    使用-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=heapdump.hprof运行
    用VisualVM进行阅读,几乎看不出(无论如何对我来说)以上堆栈跟踪中的char []充满了70%以上。 Eclipse MAT不会加载它。

    JVM args:
    CATALINA_OPTS="-server -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=heapdump.hprof -XX:MaxPermSize=1024m -XX:MaxNewSize=256m -XX:NewSize=256m -Xms768m -Xmx1024m -XX:SurvivorRatio=128 -XX:MaxTenuringThreshold=0 -XX:+UseTLAB -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSIncrementalMode -XX:-UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent
    

    压力测试:
    该应用程序可以运行几个激烈的查询,多达200个模拟JMeter用户多次遇到这些查询而没有问题……实际上,数据库确实有点慢了;)但是内存问题并未在上面复制很多次。

    因此,如果有人读过这么远,我有2条线索(可以打开更多线索):
  • Java版本-旨在退出OpenJDK并转为标准发行版
  • 我的Grails设置中发生错误时导致无限/递归处理的某件事。

  • 直到我找到能使这些机器中的鬼魂破坏的质子包,该应用程序才能上线。

    有任何想法吗?

    干杯...

    更新#1

    安装了Java 1.7并删除了OpenJDK,这肯定解决了JavaMelody显示问题。现在,它是一个等待的游戏,以查看这是否解决了主要问题。有趣的是,该网站似乎速度更快。

    最佳答案

    如果用户多次重新提交登录表单(至j_spring_security_check),则导致在自定义UserDetailsS​​ervice中调用了User.save()(例如,更新了上次登录时间),则在每个用户的后续调用中都关闭了Hibernate session 。

    将User.save()移到AuthenticationEventListener.onApplicationEvent(AuthenticationSuccessEvent事件),它可以正常工作。

    基本设置来自Spring security core plugin - events上的文档,我的类似于:

    在您的配置中添加:grails.plugin.springsecurity.useSecurityEventListener = true
    在resources.groovy中:

    import security.AuthenticationEventListener
    ....
    authenticationEventListener(AuthenticationEventListener)
    

    AuthenticationEventListener.groovy:
    class AuthenticationEventListener implements ApplicationListener<AuthenticationSuccessEvent> {
    
        void onApplicationEvent(AuthenticationSuccessEvent event) {
            if (event instanceof AuthenticationSuccessEvent) {
                UserDetails userDetails = (UserDetails) event.getAuthentication().getPrincipal()
                def httpSession = SecurityRequestHolder.request.session
    
                if (!httpSession.loggedIn) {
                    httpSession.loggedIn = true
                    synchronized (httpSession.loggedIn) {
                        httpSession.timeZone = userDetails.timeZone
                        User.withSession { session ->
                            if (session.isOpen()) {
                                User user = User.findByUsername(userDetails.username, [fetch: [roles: 'eager']])
                                user.lastLoggedIn = new Date()
                                user.save(flush: true)
                            }
                        }
                    }
                }
            }
        }
    }
    

    关于grails - Grails堆空间间歇性溢出,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/12064622/

    相关文章:

    grails - grails 3 taglib,html builder和createLInk

    mongodb - 使用Mongodb的持久JSON响应

    grails - Grails commandObject beforeValidate

    grails - 在这种情况下,grails过滤器是否应该向前运行两次?

    java - 如何从 src 文件夹访问 Hibernate session ?

    grails - 如何使用Grails GORM获得不同的结果

    sql - Grails仪表板说我没有服务

    grails - 显示闪现信息

    grails - 如何在 Grails REST Controller 中捕获异常

    grails - 在一个GebReportingSpec中驱动两个不同的浏览器?