python - 使用cursor.fetchall()返回空数据框

Question

我有一个 SQL 查询，它可能返回空或非空记录。我需要在 Python 中执行此查询:

import os
import pandas as pd
import pymysql
from pandas import ExcelWriter
from sqlalchemy import create_engine

connection = pymysql.connect(host='localhost',
                            user='root',
                            password='root',
                            db='device_db',
                            cursorclass=pymysql.cursors.DictCursor)

cur3=connection.cursor()

cur3.execute('SELECT outgoing_db.Customer_No, sum(SUM_IC_CALLS) AS SumOfSUM_IC_CALLS, sum(SUM_AIC_DURATION_C1) AS SumOfSUM_AIC_DURATION_C1, sum(SUM_AIC_AMOUNT_C1) AS SumOfSUM_AIC_AMOUNT_C1, outgoing_db.Rate, outgoing_db.Customer_Name FROM outgoing_db WHERE (((outgoing_db.User_ID) != ("IND")) AND ((outgoing_db.Incoming_ID)="Airtel") AND ((outgoing_db.Customer_Type)="Check") AND ((outgoing_db.Current_Operation)="NO") AND ((outgoing_db.Active)="NO")) GROUP BY outgoing_db.Customer_No, outgoing_db.Rate, outgoing_db.Customer_Name HAVING (((outgoing_db.Customer_No) In (33165,33197)));')
mydf3=pd.DataFrame(cur3.fetchall())
print(mydf3)

这给了我一个异常(exception):

Traceback (most recent call last):
  File "C:\wamp64\www\DevicePortal\Query.py", line 16, in <module>
    mydf3=pd.DataFrame(cur3.fetchall())
  File "C:\Users\Sonal\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\frame.py", line 404, in __init__
raise ValueError('DataFrame constructor not properly called!')
ValueError: DataFrame constructor not properly called! 

当我做 cur3.fetchone() 时，它给了我一个输出:

Empty DataFrame
Columns: []
Index: []

但 future 可能会添加数据(大约 100 万)。 是否有使用 cur3.fetchall()

返回空数据框的解决方案

Answer 1

fetchall()返回元组列表；如果你想从它们构建一个数据框，你可以使用 pandas.DataFrame.from_records()

更简单的方法是直接从 SQL 读取数据:

pandas.read_sql()

它有一个chunksize如果您的数据集太大，该参数允许您按 block 加载数据。

在这里阅读:

https://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.DataFrame.from_records.html https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql.html