使用 Flask 处理超大文件上传(1 GB 以上)的最佳方式是什么？
我的应用程序本质上是为多个文件分配一个唯一的文件编号，然后根据用户选择的位置将其保存在服务器上。
我们如何将文件上传作为后台任务运行，这样用户就不会让浏览器旋转 1 小时，而是可以立即进入下一页？
- Flask 开发服务器能够处理大量文件(50gb 需要 1.5 小时，上传速度很快，但将文件写入空白文件非常慢)
- 如果我用 Twisted 包装应用程序，应用程序会在处理大文件时崩溃
- 我试过将 Celery 与 Redis 结合使用，但这似乎不适用于已发布的上传
- 我在 Windows 上，网络服务器的选项较少
最佳答案
我认为解决问题的 super 简单方法就是将文件分成许多小部分/ block 。因此，将有两个部分来完成这项工作，即前端(网站)和后端(服务器)。
对于前端部分，您可以使用类似 Dropzone.js 的东西，它没有额外的依赖项并且包含不错的 CSS。您所要做的就是将 dropzone 类添加到表单中，它会自动将其变成它们的特殊拖放字段之一(您也可以单击并选择)。
但是，默认情况下，dropzone 不会分 block 文件。幸运的是，启用它真的很容易。
这是启用了 DropzoneJS 和 chunking 的示例文件上传表单:
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.4.0/min/dropzone.min.css"/>
<link rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.4.0/min/basic.min.css"/>
<script type="application/javascript"
src="https://cdnjs.cloudflare.com/ajax/libs/dropzone/5.4.0/min/dropzone.min.js">
</script>
<title>File Dropper</title>
</head>
<body>
<form method="POST" action='/upload' class="dropzone dz-clickable"
id="dropper" enctype="multipart/form-data">
</form>
<script type="application/javascript">
Dropzone.options.dropper = {
paramName: 'file',
chunking: true,
forceChunking: true,
url: '/upload',
maxFilesize: 1025, // megabytes
chunkSize: 1000000 // bytes
}
</script>
</body>
</html>
这是使用 flask 的后端部分:
import logging
import os
from flask import render_template, Blueprint, request, make_response
from werkzeug.utils import secure_filename
from pydrop.config import config
blueprint = Blueprint('templated', __name__, template_folder='templates')
log = logging.getLogger('pydrop')
@blueprint.route('/')
@blueprint.route('/index')
def index():
# Route to serve the upload form
return render_template('index.html',
page_name='Main',
project_name="pydrop")
@blueprint.route('/upload', methods=['POST'])
def upload():
file = request.files['file']
save_path = os.path.join(config.data_dir, secure_filename(file.filename))
current_chunk = int(request.form['dzchunkindex'])
# If the file already exists it's ok if we are appending to it,
# but not if it's new file that would overwrite the existing one
if os.path.exists(save_path) and current_chunk == 0:
# 400 and 500s will tell dropzone that an error occurred and show an error
return make_response(('File already exists', 400))
try:
with open(save_path, 'ab') as f:
f.seek(int(request.form['dzchunkbyteoffset']))
f.write(file.stream.read())
except OSError:
# log.exception will include the traceback so we can see what's wrong
log.exception('Could not write to file')
return make_response(("Not sure why,"
" but we couldn't write the file to disk", 500))
total_chunks = int(request.form['dztotalchunkcount'])
if current_chunk + 1 == total_chunks:
# This was the last chunk, the file should be complete and the size we expect
if os.path.getsize(save_path) != int(request.form['dztotalfilesize']):
log.error(f"File {file.filename} was completed, "
f"but has a size mismatch."
f"Was {os.path.getsize(save_path)} but we"
f" expected {request.form['dztotalfilesize']} ")
return make_response(('Size mismatch', 500))
else:
log.info(f'File {file.filename} has been uploaded successfully')
else:
log.debug(f'Chunk {current_chunk + 1} of {total_chunks} '
f'for file {file.filename} complete')
return make_response(("Chunk upload successful", 200))
关于python - 使用 Flask 处理大文件上传，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/44727052/