我正在尝试在 ANTLR 版本 4 的帮助下解析日志文件。目前该文件由 10703 行组成,可能超过数百万行。 Java提示以下heap Exception。我将行数减少到 300 行。程序成功运行,但是当我将其行数设置为 400 时,它又开始出现堆异常。我也放大了java的堆内存,但没有成功。我也不知道这个问题是java的还是ANTLR的。
这是执行语法的程序:
public class parser {
public static void main(String[] args) {
System.out.println("Start");
String fileName = "D:\\folder\\logs.out";
File file = new File(fileName);
FileInputStream fis = null;
try {
// Open the input file stream
fis = new FileInputStream(file);
// Create a CharStream that reads from standard input
ANTLRInputStream input = new ANTLRInputStream(fis);
GrammarOSBLexer lexer=new GrammarOSBLexer(input);
TokenStream tokenStream=new CommonTokenStream(lexer);
GrammarOSBParser parsr=new GrammarOSBParser(tokenStream);
fis.close();
try {
parsr.logs();
} catch (RecognitionException e) {
e.printStackTrace();
}
System.out.println("done!");
}catch (IOException e) {
e.printStackTrace();
}
}
}
`
这里是异常(exception):
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Arrays.copyOf(Unknown Source)
at java.util.ArrayList.grow(Unknown Source)
at java.util.ArrayList.ensureExplicitCapacity(Unknown Source)
at java.util.ArrayList.ensureCapacityInternal(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:146)
at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:137)
at org.antlr.runtime.CommonTokenStream.consume(CommonTokenStream.java:68)
at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:106)
at com.javadude.antlr.tutorial.GrammarOSBParser.string(GrammarOSBParser.java:1269)
at com.javadude.antlr.tutorial.GrammarOSBParser.random_messageText(GrammarOSBParser.java:350)
at com.javadude.antlr.tutorial.GrammarOSBParser.messageTextTag(GrammarOSBParser.java:219)
at com.javadude.antlr.tutorial.GrammarOSBParser.log(GrammarOSBParser.java:173)
at com.javadude.antlr.tutorial.GrammarOSBParser.logs(GrammarOSBParser.java:111)
at com.javadude.antlr.tutorial.parser.main(parser.java:91)
... 5 more
`
最佳答案
看起来将所有 token 缓冲在内存中效果不佳。因此,您应该寻找一种不进行缓冲的解决方案。 ANTLR4 带有 UnbufferedTokenStream更适合您的类(class)。另请参阅this question讨论如何使用这样的流以及它有哪些缺点(Sam Harwell + Terence Parr 做出了贡献)。
关于java.lang.OutOfMemory错误: Java heap space Exception while Parsing of Large file using ANTLR,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/40378692/