java - ANTLR java测试文件无法创建树语法对象

Question

我正在使用 ANTLR 3.x 创建一个针对 java 的解析器。我编写了解析器语法(用于创建抽象语法树，AST)和树语法(用于对 AST 执行操作)。最后，为了测试这两个语法文件，我用 Java 编写了一个测试文件。

看看下面的代码，

协议(protocol)语法

grammar protocol;
options {
      language = Java;
  output = AST;
}

tokens{ //imaginary tokens
PROT;
INITIALP;
PROC;
TRANSITIONS;
}
@header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}

@lexer::header {
  package com.javadude.antlr3.x.tutorial;
}
/*
parser rules, in lowercase letters
*/
program
    : declaration+
    ;
declaration
    :protocol
    |initialprocess
    |process
    |transitions
    ;

protocol
    :'protocol' ID ';' -> ^(PROT ID)
    ;
initialprocess
    :'pin' '=' INT ';' -> ^(INITIALP INT)
    ;
process
    :'p' '=' INT ';' -> ^(PROC INT)
    ;
transitions
    :'transitions' '=' INT ('(' INT ',' INT ')') + ';' -> ^(TRANSITIONS INT INT INT*)
    ;

/*
lexer rules (tokens), in upper case letters
*/
ID  
    : (('a'..'z' | 'A'..'Z'|'_')('a'..'z' | 'A'..'Z'|'0'..'9'|'_'))*;
INT 
    : ('0'..'9')+;
WHITESPACE
    : ('\t' | ' ' | '\r' | '\n' | '\u000C')+ {$channel = HIDDEN;};

协议(protocol)行者

grammar protocolWalker;

options {
  language = Java;
  //Error, eclipse can't access tokenVocab named protocol
  tokenVocab = protocol;    //import tokens from protocol.g i.e, from protocol.tokens file
  ASTLabelType = CommonTree;
  }

@header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}

program
    : declaration+
    ;

declaration
    :protocol
    |initialprocess
    |process
    |transitions
    ;

protocol
    :^(PROT ID)
    {System.out.println("create protocol " +$ID.text);}
    ;

initialprocess
    :^(INITIALP INT)
    {System.out.println("");}
    ;

process
    :^(PROC INT)
    {System.out.println("");}
    ;

transitions
    :^(TRANSITIONS INT INT INT*)
    {System.out.println("");}
    ;

协议(protocol)测试.java

package com.javadude.antlr3.x.tutorial;  
import org.antlr.runtime.*;  
import org.antlr.runtime.tree.*;  
import org.antlr.runtime.tree.CommonTree;  
import org.antlr.runtime.tree.CommonTreeNodeStream;  
public class Protocoltest { 



/**
 * @param args
 */
public static void main(String[] args) throws Exception {
    //create input stream from standard input
    ANTLRInputStream input = new ANTLRInputStream(System.in);
    //create a lexer attached to that input stream
    protocolLexer lexer = new protocolLexer(input);
    //create a stream of tokens pulled from the lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer);

    //create a pareser attached to teh token stream
    protocolParser parser = new protocolParser(tokens);
    //invoke the program rule in get return value
    protocolParser.program_return r =parser.program();
    CommonTree t = (CommonTree)r.getTree();
    //output the extracted tree to the console
    System.out.println(t.toStringTree());

    //walk resulting tree; create treenode stream first
    CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
    //AST nodes have payloads that point into token stream
    nodes.setTokenStream(tokens);


    //create a tree walker attached to the nodes stream  
            //Error, can't create TreeGrammar object called walker
    protocolWalker walker = new protocolWalker(nodes);

    //invoke the start symbol, rule program
    walker.program();
    }
}

问题:

1. 在protocolWalker中，我无法访问 token (protocol.tokens)

   //Error, eclipse can't access tokenVocab named protocol  
       tokenVocab = protocol; //import tokens from protocol.g i.e, from protocol.tokens file
   

2. 在protocolWalker中，我可以在操作列表中创建名为Configuration的java类对象吗？

   protocol
       :^(PROT ID)
          {System.out.println("create protocol " +$ID.text);
           Configuration conf = new Configuration();
          }
       ;
   

3. 在 Protocoltest.java 中

   //create a tree walker attached to the nodes stream    
   //Error, can't create TreeGrammar object called walker  
       protocolWalker walker = new protocolWalker(nodes);  
   

   无法创建protocolWalker对象。我在示例和教程中看到创建了这样的对象。

Answer 1

In protocolWalker, I can't access the tokens (protocol.tokens)...

它似乎可以很好地访问protocol.tokens:将tokenVocab更改为其他内容会产生现在不会产生的错误。 protocolWalker.g 的问题在于它被定义为 token 解析器(语法 protocolWalker)，但它的使用方式类似于树解析器。将语法定义为树语法protocolWalker消除了我看到的有关未定义标记的错误。

In protocolWalker, can I create the object of java class, called Configuration, in the action list?

是的，可以。正常的 Java 编程注意事项适用于导入类等，但您可以像 System.out.println 这样的代码一样使用它。

In Protocoltest.java ... Object of protocolWalker can't be created.

protocolWalker.g(现在)生成一个名为 protocolWalkerParser 的标记解析器。当您将其更改为树语法时，它将生成一个名为 protocolWalker 的树解析器。

非常感谢您发布整个语法。这使得回答问题变得更加容易。