python - 使用 python 和 VueJS 优化 5MB 查询结果的显示

Question

我有一个 Web 应用程序，在服务器(bottle)上使用 python，在客户端使用 VueJS。 一个前端组件显示查询的分页结果，该查询以超过 10k 条记录结尾。记录保存为 python dict 并发送到前端。结果的大小约为 5.5MB，我的互联网连接结束时等待了 2 秒以上。

处理请求的 api 路径如下所示:

@cmCampaigns.get('/api/campaignManager/campaigns')
@authorize()
def get():
    resp = {}
    usr = authlayer.current_user
    user_id = dao.App().getUserID(usr.username)
    resp["campaigns"] = CampaignsData().get_revcontent_campaigns(user_id)

    return resp

查询本身是这样的:

def get_campaigns(self, user_id):
    query = "SELECT rc.account_id, rc.campaign_id, " \
            "rc.campaign_name, rc.start_date, rc.end_date, " \
            "rc.enabled, rc.default_bid, " \
            "rc.budget, rc.cost, ctr,rc.country_codes, " \
            "'revcontent' AS provider, " \
            "replace(JSON_EXTRACT(ac.account_json,'$.client_id'),'\"','') AS account_name, " \
            "CASE enabled " \
            "WHEN enabled = 'active' THEN 'on' " \
            "WHEN enabled = 'inactive' THEN 'off' " \
            "END AS enabled_val " \
            "FROM prv_campaigns AS rc " \
            "INNER JOIN websites AS ws " \
            "ON rc.website_id = ws.website_id " \
            "INNER JOIN website_users AS wu " \
            "ON wu.website_id = ws.website_id " \
            "INNER JOIN prv_accounts ac " \
            "ON rc.account_id = ac.account_id " \
            "WHERE wu.user_id = %s " \
            "ORDER BY id DESC"
    try:
        data = self.db.query(query, user_id)
        return data
    except Exception as e:
        logging.exception(e.message)
        return -1, e.message

我尝试尽可能优化查询，但仍然不够。 有哪些“最佳实践”解决方案可以优化这种通用逻辑？在发送之前压缩字典？我在考虑分页，但后来我的过滤和排序逻辑在客户端，所以很多数据显然消失了……有什么建议吗？？谢谢

Answer 1

对于分页，您可以使用 LIMIT 和 OFFSET

优化？在查询之前运行“EXPLAIN”并带来输出

让我们快速浏览一下

SELECT rc.account_id, 
       rc.campaign_id, 
       rc.campaign_name, 
       rc.start_date, 
       rc.end_date, 
       rc.enabled, 
       rc.default_bid, 
       rc.budget, 
       rc.cost, 
       ctr, 
       rc.country_codes, 
       'revcontent'                                                   AS 
       provider, 
       Replace(Json_extract(ac.account_json, '$.client_id'), '"', '') AS 
       account_name, 
       CASE enabled 
         WHEN enabled = 'active' THEN 'on' 
         WHEN enabled = 'inactive' THEN 'off' 
       END                                                            AS 
       enabled_val 
FROM   prv_campaigns AS rc 
       inner join websites AS ws 
               ON rc.website_id = ws.website_id 
       inner join website_users AS wu 
               ON wu.website_id = ws.website_id 
       inner join prv_accounts ac 
               ON rc.account_id = ac.account_id 
WHERE  wu.user_id = %s 
ORDER  BY id DESC 

• 启用的列应该是数字或枚举，这将节省 strcmp 时间
• Replace(Json_extract(ac.account_json, '$.client_id'), '"', '') ? 提取所有账户名在客户端做你的事
• 2 个内连接 ?可能是设计问题