ANTLR : How to replace all characters defined as space with actual space

Question

我的ANTLR代码如下:

LPARENTHESIS : ('('); 
RPARENTHESIS : (')'); 

fragment CHARACTER : ('a'..'z'|'0'..'9'|); 
fragment QUOTE     : ('"'); 
fragment WILDCARD  : ('*'); 
fragment SPACE     : (' '|'\n'|'\r'|'\t'|'\u000C'|';'|':'|','); 

WILD_STRING 
   : (CHARACTER)* 
     ( 
       ('?') 
       (CHARACTER)* 
     )+ 
   ; 
PREFIX_STRING 
   : (CHARACTER)+
     ( 
       ('*')  
     )+ 
   ; 
WS     : (SPACE) { $channel=HIDDEN; }; 
PHRASE : (QUOTE)(LPARENTHESIS)?(WORD)(WILDCARD)?(RPARENTHESIS)?((SPACE)+(LPARENTHESIS)?(WORD)(WILDCARD)?(RPARENTHESIS)?)*(SPACE)+(QUOTE); 
WORD   : (CHARACTER)+; 

我想要做的是将所有标记为空格的字符替换为 PHRASE 中的实际空格字符。另外，如果可能的话，我希望所有连续空间都由单个空格表示。

任何帮助将不胜感激。由于某种原因，我发现很难理解 ANTLR。有什么好的教程吗？

Answer 1

Java

调用您的lexer's setText(...) method :

grammar T;

parse
  :  words EOF {System.out.println($words.text);}
  ;

words    
  :  Word (Spaces Word)* 
  ;

Word  
  :  ('a'..'z'|'A'..'Z')+
  ;

Spaces
  :  (' ' | '\t' | '\r' | '\n')+ {setText(" ");}
  ;

可以使用以下类进行测试:

import org.antlr.runtime.*;

public class Main {
    public static void main(String[] args) throws Exception {
        String source = "This         is     \n    just \t\t\t\t\t\t a \n\t\t test";
        ANTLRStringStream in = new ANTLRStringStream(source);
        TLexer lexer = new TLexer(in);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        TParser parser = new TParser(tokens);
        System.out.println("------------------------------\nSource:\n" + source +
                "\n------------------------------\nAfter parsing:");
        parser.parse();
    }
}

产生以下输出:

------------------------------
Source:
This         is     
    just                         a 
         test
------------------------------
After parsing:
This is just a test

Puneet Pawaia wrote:

Any help would be most appreciated. For some reason, I am finding it hard to understand ANTLR. Any good tutorials out there ?

ANTLR Wiki有大量信息，尽管有点非结构化(但这可能就是我!)。

最好的 ANTLR 教程是这本书:The Definitive ANTLR Reference: Building Domain-Specific Languages .

C#

对于 C# 目标，请尝试以下操作:

grammar T;

options {
  language=CSharp2;
}

@parser::namespace { Demo }
@lexer::namespace { Demo }

parse
  :  words EOF {Console.WriteLine($words.text);}
  ;

words    
  :  Word (Spaces Word)* 
  ;

Word  
  :  ('a'..'z'|'A'..'Z')+
  ;

Spaces
  :  (' ' | '\t' | '\r' | '\n')+ {Text = " ";}
  ;

与测试类:

using System;
using Antlr.Runtime;

namespace Demo
{
    class MainClass
    {
        public static void Main (string[] args)
        {
            ANTLRStringStream Input = new ANTLRStringStream("This         is     \n    just \t\t\t\t\t\t a \n\t\t test"); 
            TLexer Lexer = new TLexer(Input);
            CommonTokenStream Tokens = new CommonTokenStream(Lexer);
            TParser Parser = new TParser(Tokens);
            Parser.parse();
        }
    }
}

它还会将 This is just a test 打印到控制台。我尝试使用 SetText(...) 而不是 setText(...) 但这也不起作用，并且 C# API docs目前处于离线状态，所以我使用了试错法 {Text = "";}。我用C# 3.1.1 runtime DLL's测试了它.

祝你好运!