c++ - 根据 C++ 标准的语法解析数字文字的不一致

Question

通读 C++17 标准，在我看来，预处理器处理的 pp-number 与数字文字之间存在不一致，例如user-defined-integer-literal，因为它们被定义为由“上层”语言处理。

例如，以下根据预处理器语法正确解析为pp-number:

123_e+1

但放置在符合 C++11 的代码片段的上下文中，

int  operator"" _e(unsigned long long)
    { return 0; }

int test()
    {
    return 123_e+1;
    }

当前的 Clang 或 GCC 编译器(我还没有测试过其他编译器)会返回类似这样的错误:

unable to find numeric literal operator 'operator""_e+1'

其中 operator""_e(...) 未找到，尝试定义 operator""_e+1(...) 将无效。

这似乎是因为编译器首先将标记作为 pp-number 进行词法分析，但随后无法回滚并应用语法规则user-defined-integer-literal 解析最终表达式时。

相比之下，以下代码编译得很好:

int  operator"" _d(unsigned long long)
    { return 0; }

int test()
    {
    return 0x123_d+1;  // doesn't lex as a 'pp-number' because 'sign' can only follow [eEpP]
    }

这是对标准的正确解读吗？如果是这样，编译器应该处理这种可以说是罕见的极端情况是否合理？

Answer 1

您已成为 maximal munch rule 的受害者它使词法分析器采用尽可能多的字符来形成有效的标记。

这在 [lex.pptoken]p3 部分中有介绍上面写着(强调我的):

Otherwise, the next preprocessing token is the longest sequence of characters that could constitute a preprocessing token, even if that would cause further lexical analysis to fail, except that a header-name ([lex.header]) is only formed within a #include directive.

并包括几个例子:

[ Example:

#define R "x"
const char* s = R"y";           // ill-formed raw string, not "x" "y"

— end example ]

4 [ Example: The program fragment 0xe+foo is parsed as a preprocessing number token (one that is not a valid floating or integer literal token), even though a parse as three preprocessing tokens 0xe, +, and foo might produce a valid expression (for example, if foo were a macro defined as 1). Similarly, the program fragment 1E1 is parsed as a preprocessing number (one that is a valid floating literal token), whether or not E is a macro name. — end example ]

5[ Example: The program fragment x+++++y is parsed as x ++ ++ + y, which, if x and y have integral types, violates a constraint on increment operators, even though the parse x ++ + ++ y might yield a correct expression. — end example  ]

此规则适用于其他几个众所周知的案例，例如 a+++++b和 tokens >= which required a fix to allow .

pp-token语法引用如下:

pp-number:  
  digit  
  . digit  
  pp-number digit  
  pp-number identifier-nondigit 
  pp-number ' digit  
  pp-number ' nondigit    
  pp-number e sign  
  pp-number E sign  
  pp-number p sign  
  pp-number P sign  
  pp-number .  

注意 e 符号 的产生，这就是这个案例的障碍。另一方面，如果您像第二个示例一样使用 d ，则不会遇到此问题( see it live on godbolt )。

此外，添加间距也可以解决您的问题，因为您将不再受制于最大咀嚼 (see it live on godbolt):

123_e + 1