python - 带有编解码器 header 的 ASCII 安全文件的编码问题，具体取决于行数

Question

这是 Windows 上 Python 3.5.2 的神奇错误，这让我很沮丧。以下文件在此系统上失败:

C:\Python35\python.exe encoding-problem-cp1252.py
 File "encoding-problem-cp1252.py", line 2
SyntaxError: encoding problem: cp1252

几乎不包含任何内容 - 除了 coding header 之外还有一堆空行，但是删除任何行，即使是空行，它也会再次工作。我认为这是一个本地问题，所以我设置了 job on AppVeyor表现出相同的行为。

Python 是怎么回事？

有一个binary accurate version以下文件:

#!/usr/bin/env python
# -*- coding: cp1252 -*-


"""
There is nothing in this file, except that it is more
than 50 lines long. Running it with Python 3.5.2 on
Windows gives the following error:

    >python encoding-problem-cp1252.py
      File "encoding-problem-cp1252.py", line 2
    SyntaxError: encoding problem: cp1252

    >python
    Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (Intel)] on win32
    Type "help", "copyright", "credits" or "license" for more information.

If you remove any lines from this file, it will
execute successfully.
"""



def restore(dump):
  """













  """
  return



def main():
  print('ok')



if __name__ == '__main__':
  main()

Answer 1

这看起来像是由 issue #20731 引起的回归.看起来位置计算假设总是有 CRLF 行结尾，而您的文件只有 LF 字符，导致不正确的偏移量 being calculated here :

fd = fileno(tok->fp);
/* Due to buffering the file offset for fd can be different from the file
 * position of tok->fp.  If tok->fp was opened in text mode on Windows,
 * its file position counts CRLF as one char and can't be directly mapped
 * to the file offset for fd.  Instead we step back one byte and read to
 * the end of line.*/
pos = ftell(tok->fp);
if (pos == -1 ||
    lseek(fd, (off_t)(pos > 0 ? pos - 1 : pos), SEEK_SET) == (off_t)-1) {
    PyErr_SetFromErrnoWithFilename(PyExc_OSError, NULL);
    goto cleanup;
}

当您将文件转换为使用 Windows (CRLF) 行结尾时，问题就消失了，但我可以理解，对于跨平台脚本来说，这不是一个实用的解决方案。

我已经提交了 issue #27797 ;这应该在 Python 本身中修复。