django - 尝试使用 Redis 队列在 heroku 上的 django 中进行一些异步处理

Question

我正在尝试将更大的文件异步上传到 s3。我已经关注了来自 http://python-rq.org/ 的文档. Worker 正在运行，但没有完成工作。

View .py

 def youmaterial(request):

    if request.method == 'POST':

        newdoc = Youmaterial(docfile = request.FILES['file'],user=request.user)
        newdoc.save()
        msg='dee'
            # Redirect to the document list after POST
        return HttpResponse(json.dumps({'message': msg}))

    else:
        form1 = DocumentForm()
    return render(request,'mat_upload.html',{'form':form1})

def create_job():
    redis_conn = Redis()
    q = Queue(connection=redis_conn)  # no args implies the default queue

# Delay execution of count_words_at_url('http://nvie.com')
    job = q.enqueue(youmaterial, 'http://heroku.com')

worker .py

import os
import urlparse
from redis import Redis
from rq import Worker, Queue, Connection

listen = ['high', 'default', 'low']

redis_url = os.getenv('REDISTOGO_URL')
if not redis_url:
    raise RuntimeError('Set up Redis To Go first.')

urlparse.uses_netloc.append('redis')
url = urlparse.urlparse(redis_url)
conn = Redis(host=url.hostname, port=url.port, db=0, password=url.password)

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

过程:

web: gunicorn gettingstarted.wsgi --log-file -
worker: python -u worker.py

Settings.py(这给了我错误。)

DJANGO_SETTINGS_MODULE=config.settings rqworker high default low  

Answer 1

AFAIK，您不能以这种方式对整个 View 函数进行排队(需要一个 request 对象，而您传递一个 str)。

您是否正在尝试实现某种异步处理 View ？相反，您应该确定长处理函数并将作业放入 View 中，更像这样:

def s3_async_upload(obj):
    obj.save()

def youmaterial(request):

    if request.method == 'POST':

        newdoc = Youmaterial(docfile=request.FILES['file'], user=request.user)
        # Replace newdoc.save()
        q.enqueue(s3_async_upload, obj=newdoc)
        msg = 'dee'
        # Redirect to the document list after POST
        return HttpResponse(json.dumps({'message': msg}))

    else:
        form1 = DocumentForm()

    return render(request, 'mat_upload.html', {'form': form1})

# or with the `@job` decorator, usage: s3_async_upload(obj=newdoc)

@job
def s3_async_upload(obj):
    obj.save()

也许其他设置会丢失，但希望对您有所帮助!