查询时 Neo4J OutOfMemory 错误

Question

我最近开始使用 Neo4J，但到目前为止我还没有找到我所遇到问题的答案，尤其是服务器方面的问题。我正在使用 1.8.1 版并将服务器作为 Windows 上的服务运行，而不是嵌入式。我的图表有大约 700 万个节点和近 1100 万个关系。

对于小查询和倍数，事情运行得很好。但是，当我试图拉回更复杂的查询(可能有数千行)时，事情就变糟了。如果我正在使用控制台，我什么也得不到，然后在几分钟或更长时间后出现未定义一词(它试图用 Javascript 做一些事情，但我不确定是什么)。如果我在 .NET 中使用 Neo4JClient，它将超时(我正在通过 WCF 服务处理此问题)并且我怀疑我的问题出在服务器端。

这是一个在控制台中导致我出现问题的示例密码查询:

start begin = node:idx(ID="1234")
MATCH begin-[r1?:RELATED_TO]-n1-[r2?:RELATED_TO]-n2-[r3?:RELATED_TO]-n3-[r4?:RELATED_TO]-n4
RETURN begin.Title?, r1.RelationType?, n1.Title?, r2.RelationType?, n2.Title?, r3.RelationType?, n3.Title?, r4.RelationType?, n4.Title?;

我查看了日志并收到以下严重错误:

SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at java.io.StringWriter.write(Unknown Source)
    at java.io.PrintWriter.newLine(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at java.io.PrintWriter.println(Unknown Source)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult$$anonfun$dumpToString$1.apply(PipeExecutionResult.scala:96)
    at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
    at scala.collection.immutable.List.foreach(List.scala:45)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:96)
    at org.neo4j.cypher.PipeExecutionResult.dumpToString(PipeExecutionResult.scala:124)
    at org.neo4j.cypher.javacompat.ExecutionResult.toString(ExecutionResult.java:90)
    at org.neo4j.shell.kernel.apps.Start.exec(Start.java:72)
    at org.neo4j.shell.kernel.apps.ReadOnlyGraphDatabaseApp.execute(ReadOnlyGraphDatabaseApp.java:32)
    at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:127)
    at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:92)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:130)
    at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:114)
    at org.neo4j.server.webadmin.rest.ShellSession.evaluate(ShellSession.java:96)
    at org.neo4j.server.webadmin.rest.ConsoleService.exec(ConsoleService.java:123)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)

从有根据的猜测角度来看堆栈跟踪，是不是我拉回了太多记录？因为它在扩展 StringBuffer 时内存不足。

我想知道 GC 是否可以发挥作用，所以我找到了 GCViewer。它似乎不是垃圾收集，如果您认为它有用，我可以添加 GCViewer 的屏幕截图。

我已经为 JVM 分配了默认值和 8G 内存之间的任意值。这是我的配置文件中的一些设置(我会尝试只包含相关的设置)。如果您还需要，请告诉我。

Neo4J.properties

# Default values for the low-level graph engine
use_memory_mapped_buffers=false

# Keep logical logs, helps debugging but uses more disk space, enabled for legacy reasons
keep_logical_logs=true

Neo4J-server.properties

# HTTP logging is disabled. HTTP logging can be enabled by setting this property to 'true'.
org.neo4j.server.http.log.enabled=false 

Neo4J-Wrapper.conf(可能不熟练地放在一起)

# Uncomment the following line to enable garbage collection logging
wrapper.java.additional.4=-Xloggc:data/log/neo4j-gc.log

# Setting a different Garbage Collector as recommended by Neo4J
wrapper.java.additional.5=-XX:+UseConcMarkSweepGC

# other beneficial settings that should boost performance
wrapper.java.additional.6=-d64
wrapper.java.additional.7=-server
wrapper.java.additional.8=-Xss1024k

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=1024

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=8000

如有任何帮助，我们将不胜感激。

Answer 1

您的查询太复杂了。当你有一个如此大的图而不是确保你不会达到你的堆内存限制时，你必须分配适当的内存。你可能想尝试一下这个配置: io examples .

但是，您的查询可以简化为:

start begin = node:idx(ID="1234")
MATCH p=begin-[r1:RELATED_TO*0..4]-n4
RETURN p