我们已经设置了一个 Spring Batch 批处理电子邮件发送器作业，它执行以下操作:
- 从 JMS 队列中读取传入电子邮件的详细信息。
- 不做任何处理，只是让它们通过...
- 将详细信息变成电子邮件并将它们“写入”到 SMTP
- 无限重复
这通常工作正常，但我们发现了一个巨大的内存泄漏。大约 9 小时后，可以看到一个 FlowJob，但有 748 个 JobExecution(全部“已启动”)，每个包含 778 (!) StepExecution实例。总而言之，900 MB 的内容。
这是 Spring 配置(Spring 3.1，Spring Batch 2.1.9):
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
<property name="taskExecutor">
<bean class="org.springframework.core.task.SimpleAsyncTaskExecutor"/>
</property>
</bean>
<bean id="jmsEmailFetcher" class="org.springframework.batch.item.jms.JmsItemReader">
<property name="jmsTemplate" ref="batchEmailJmsTemplate" />
</bean>
<bean id="passthroughProcessor" class="org.springframework.batch.item.support.PassThroughItemProcessor" />
<bean id="transactionManager"
class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<!-- The Spring Batch *Limiter/Decider* -->
<bean id="ourLimitDecider" class="our.special.InfiniteThrottledExecutionDecider" />
<!-- The Spring Batch *Job* -->
<batch:job id="fetchEmailsJobBatch" restartable="true">
<batch:step id="fetchEmailsStep" next="limitDecision">
<batch:tasklet throttle-limit="10">
<batch:chunk reader="jmsEmailFetcher" processor="passthroughProcessor"
writer="batchEmailService.javaMailItemWriter" commit-interval="100" skip-limit="999999999"> <!-- Might gets *lots* of MQ-not-up-yet fails -->
<batch:skippable-exception-classes>
<batch:include class="org.springframework.jms.JmsException" /> <!-- Generally MQ-not-up-yet ConnectionException, or Session-closed (in tests) -->
<batch:include class="java.lang.IllegalStateException" /> <!-- Yuk, usually/presumably a test SMTP server isn't yet connected -->
<batch:include class="org.springframework.mail.MailSendException" /> <!-- SMTP down... -->
</batch:skippable-exception-classes>
</batch:chunk>
</batch:tasklet>
</batch:step>
<batch:decision id="limitDecision" decider="ourLimitDecider">
<batch:next on="CONTINUE" to="fetchEmailsStep" />
<batch:end on="COMPLETED" />
</batch:decision>
</batch:job>
我们的 InfiniteThrottledExecutionDecider 基本上每次都返回 new FlowExecutionStatus("CONTINUE")，以确保 fetchEmailsStep 在结束时执行流，并且 Job 永远不会完成 - 至少在我们准备好自己停止它之前不会。
我们不使用数据库，所以我们希望一些东西保存在内存中，但不是所有运行过的完整记录...
我们的配置有问题吗？还是我们的方法？
这是我们的日志文件中的一些内容，我们被告知的是步骤 #778 以及应该是唯一的作业实例。
23:58:18,782 - INFO (org.springframework.batch.core.job.SimpleStepHandler) - Duplicate step [fetchEmailsStep] detected in execution of job=[fetchEmailsJobBatch]. If either step fails, both will be executed again on restart.
23:59:52,257 - INFO (org.springframework.batch.core.job.SimpleStepHandler) - Executing step: [fetchEmailsStep]
23:59:52,257 - DEBUG (org.springframework.batch.core.step.AbstractStep) - Executing: id=778
23:59:52,259 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Starting repeat context.
23:59:52,259 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Repeat operation about to start at count=1
23:59:52,259 - DEBUG (org.springframework.batch.core.scope.context.StepContextRepeatCallback) - Preparing chunk execution for StepContext: org.springframework.batch.core.scope.context.StepContext@1be1ee
23:59:52,259 - DEBUG (org.springframework.batch.core.scope.context.StepContextRepeatCallback) - Chunk execution starting: queue size=0
23:59:52,259 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Starting repeat context.
23:59:52,259 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Repeat operation about to start at count=1
... 5 second JMS timeout ...
23:59:57,716 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Repeat is complete according to policy and result value.
23:59:57,716 - DEBUG (org.springframework.batch.core.step.item.ChunkOrientedTasklet) - Inputs not busy, ended: true
23:59:57,716 - DEBUG (org.springframework.batch.core.step.tasklet.TaskletStep) - Applying contribution: [StepContribution: read=0, written=0, filtered=0, readSkips=0, writeSkips=0, processSkips=0, exitStatus=EXECUTING]
23:59:57,719 - DEBUG (org.springframework.batch.core.step.tasklet.TaskletStep) - Saving step execution before commit: StepExecution: id=778, version=1, name=fetchEmailsStep, status=STARTED, exitStatus=EXECUTING, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0, exitDescription=
23:59:57,721 - DEBUG (org.springframework.batch.repeat.support.RepeatTemplate) - Repeat is complete according to policy and result value.
23:59:57,721 - DEBUG (org.springframework.batch.core.step.AbstractStep) - Step execution success: id=778
23:59:57,722 - DEBUG (org.springframework.batch.core.step.AbstractStep) - Step execution complete: StepExecution: id=778, version=3, name=fetchEmailsStep, status=COMPLETED, exitStatus=COMPLETED, readCount=0, filterCount=0, writeCount=0 readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1, rollbackCount=0
23:59:57,723 - DEBUG (org.springframework.batch.core.job.flow.support.SimpleFlow) - Completed state=fetchEmailsJobBatch.fetchEmailsStep with status=COMPLETED
23:59:57,723 - DEBUG (org.springframework.batch.core.job.flow.support.SimpleFlow) - Handling state=fetchEmailsJobBatch.limitDecision100
23:59:57,723 - DEBUG (org.springframework.batch.core.job.flow.support.SimpleFlow) - Completed state=fetchEmailsJobBatch.limitDecision100 with status=CONTINUE
23:59:57,723 - DEBUG (org.springframework.batch.core.job.flow.support.SimpleFlow) - Handling state=fetchEmailsJobBatch.fetchEmailsStep
问题是，堆转储显示 748 个 JobExecution 和 581,911 个 StepExecution。它们都来自哪里？
最佳答案
好的，在回答我的问题时，解决方案是删除步骤流程并让“fetchEmailsStep”永远运行。
我们通过删除 JmsTemplate 实现了这一点(“batchEmailJmsTemplate”) 的“readTimeout”，以及子类化 org.springframework.batch.item.jms.JmsItemReader删除它阻止你使用 infinite JmsTemplate 的检查s - 并添加额外的日志记录。如果您不想覆盖，可以将“readTimeout”设置为一个很大的数字。
我仍然不知道为什么内存使用量如此之大...
[编辑] 我也在使用 JobExecutionDecider ("our.special.InfiniteThrottledExecutionDecider") 来实现节流(基于 <StepExecution>.writeCount )，但这不得不去。我们现在使用扩展 StepExecutionListenerSupport 的新类实现了相同的结果。 (或者你可以实现 StepExecutionListener )。此 bean 的实例添加为 <batch:listener>到我们的单<batch:step>
关于java - FlowJob 的 Spring Batch 内存泄漏，重复步骤，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/13431570/