python - 出现错误 : [Errno 95] Operation not supported while writing zip file in databricks

Question

在这里，我尝试使用 Databricks 中的以下代码压缩文件并将其写入一个文件夹(挂载点)。

# List all files which need to be compressed
import os
modelPath  = '/dbfs/mnt/temp/zip/'
filenames = [os.path.join(root, name) for root, dirs, files in os.walk(top=modelPath , topdown=False) for name in files]
print(filenames)

zipPath = '/dbfs/mnt/temp/compressed/demo.zip'
import zipfile
with zipfile.ZipFile(zipPath, 'w') as myzip:
  for filename in filenames:
    print(filename)
    print(myzip)
    myzip.write(filename)

但我收到错误 [Errno 95] 操作不受支持。

错误详细信息

OSError                                   Traceback (most recent call last)
<command-2086761864237851> in <module>
     15     print(myzip)
---> 16     myzip.write(filename)

/usr/lib/python3.8/zipfile.py in write(self, filename, arcname, compress_type, compresslevel)
   1775             with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
-> 1776                 shutil.copyfileobj(src, dest, 1024*8)
   1777 

/usr/lib/python3.8/zipfile.py in close(self)
   1181                 self._fileobj.write(self._zinfo.FileHeader(self._zip64))
-> 1182                 self._fileobj.seek(self._zipfile.start_dir)
   1183 

OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/usr/lib/python3.8/zipfile.py in close(self)
   1837                     if self._seekable:
-> 1838                         self.fp.seek(self.start_dir)
   1839                     self._write_end_record()

OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
OSError: [Errno 95] Operation not supported

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<command-2086761864237851> in <module>
     14     print(filename)
     15     print(myzip)
---> 16     myzip.write(filename)

/usr/lib/python3.8/zipfile.py in __exit__(self, type, value, traceback)
   1310 
   1311     def __exit__(self, type, value, traceback):
-> 1312         self.close()
   1313 
   1314     def __repr__(self):

/usr/lib/python3.8/zipfile.py in close(self)
   1841             fp = self.fp
   1842             self.fp = None
-> 1843             self._fpclose(fp)
   1844 
   1845     def _write_end_record(self):

/usr/lib/python3.8/zipfile.py in _fpclose(self, fp)
   1951         self._fileRefCnt -= 1
   1952         if not self._fileRefCnt and not self._filePassed:
-> 1953             fp.close()
   1954 
   1955 

谁能帮我解决这个问题。

注意:这里我可以使用shutil压缩文件，但我想避免驱动程序，所以使用上面的方法。

Answer 1

您没有提供装载的详细信息，可能是 Blob 存储或 ADLSv2，并且显然它不允许文件查找。

看看这个简单的代码片段:

%python

path = '/dbfs/mnt/temp/testfile'

with open(path, "w") as f:
    f.write("test")
    f.seek(1)
    f.write("x")

with open(path, "r") as f:
    print(f.read()) 

它将在 f.seek(1) 处抛出“不支持操作”。

对 path = '/tmp/testfile' 重复相同的操作，您将获得正确的结果(“txst”)。

奇怪的是，根本不应该到达 zipfile.py 中的搜索，看起来 self._seekable 返回了错误的值，我不确定这是库的问题还是 azure 。

无论如何，只需在本地目录中创建存档，然后将其移动到挂载即可。

tempPath = '/tmp/demo.zip'
zipPath = '/dbfs/mnt/temp/compressed/demo.zip'
import zipfile
import os

with zipfile.ZipFile(tempPath, 'w') as myzip:
  for filename in filenames:
    print(filename)
    print(myzip)
    myzip.write(filename)

os.rename(tempPath, zipPath)