python - 使用 python 库 rply 时，在解析多行时出现意外的标记错误。我怎样才能解决这个问题？

Question

为了练习，我决定使用一种简单的语言。当只有一行时，我的 say();命令工作正常，但是当我连续执行两个命令时，出现错误。

为了解析，我使用 rply。我正在遵循这个(https://blog.usejournal.com/writing-your-own-programming-language-and-compiler-with-python-a468970ae6df)指南。我进行了大量搜索，但找不到解决方案。

这是Python代码:

from rply import ParserGenerator
from ast import Int, Sum, Sub, Say, String


class Parser():
   def __init__(self):
        self.pg = ParserGenerator(
            # A list of all token names accepted by the parser.
            ['INTEGER', 'SAY', 'OPEN_PAREN', 'CLOSE_PAREN',
             'SEMI_COLON', 'SUM', 'SUB', 'STRING']
        )

    def parse(self):
        @self.pg.production('say : SAY OPEN_PAREN expression CLOSE_PAREN SEMI_COLON')
        def say(p):
            return Say(p[2])

        @self.pg.production('expression : expression SUM expression')
        @self.pg.production('expression : expression SUB expression')
        def expression(p):
            left = p[0]
            right = p[2]
            operator = p[1]
            if operator.gettokentype() == 'SUM':
                return Sum(left, right)
            elif operator.gettokentype() == 'SUB':
                return Sub(left, right)

        @self.pg.production('expression : INTEGER')
        def int(p):
            return Int(p[0].value)

        @self.pg.production('expression : STRING')
        def string(p):
            return String(p[0].value)

        @self.pg.error
        def error_handler(token):
            raise ValueError("Ran into a %s where it wasn't expected" % token.gettokentype())

    def get_parser(self):
        return self.pg.build()

当我使用以下输入运行程序时:

say("yo");

它工作正常并返回哟。 但是，当我输入:

say("yo");
say("yoyo");

我希望它返回 yo yoyo，但我收到此错误:

C:\Users\gdog1\Desktop\proj\intparser.py:42: ParserGeneratorWarning: 4 
shift/reduce conflicts
  return self.pg.build()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.3\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:/Users/gdog1/Desktop/proj/main.py", line 20, in <module>
    parser.parse(tokens).eval()
  File "C:\Python27\lib\site-packages\rply\parser.py", line 60, in parse
    self.error_handler(lookahead)
  File "C:\Users\gdog1\Desktop\proj\intparser.py", line 39, in error_handler
    raise ValueError("Ran into a %s where it wasn't expected" % 
token.gettokentype())
ValueError: Ran into a SAY where it wasn't expected  

Answer 1

你的语法描述了一个命令:

say : SAY OPEN_PAREN expression CLOSE_PAREN SEMI_COLON

这就是解析器接受的内容。

如果您希望输入由多个命令组成，则需要编写一个描述该输入的语法:

program : 
program : program say