python - 如何使用python并行查询数据库

标签 python database parallel-processing

我有两个用于查询数据库的函数。假设有两个单独的查询,如何并行运行它们以查询同一个数据库,并在继续执行其余代码之前等待两个结果返回?

def query1(param1, param2):
    result = None
    logging.info("Connecting to database...")
    try:
        conn = connect(host=host, port=port, database=db)
        curs = conn.cursor()
        curs.execute(query)
        result = curs
        curs.close()
        conn.close()
    except Exception as e:
        logging.error("Unable to access database %s" % str(e))
    return result


def query2(param1, param2):
    result = None 
    logging.info("Connecting to database...")
    try:
        conn = connect(host=host, port=port, database=db)
        curs = conn.cursor()
        curs.execute(query)
        result = curs
        curs.close()
        conn.close()  
    except Exception as e:
        logging.error("Unable to access database %s" % str(e))    
    return result

最佳答案

这是一个多线程代码,可以完成您要完成的任务:

from threading import Thread, Lock

class DatabaseWorker(Thread):
    __lock = Lock()

    def __init__(self, db, query, result_queue):
        Thread.__init__(self)
        self.db = db
        self.query = query
        self.result_queue = result_queue

    def run(self):
        result = None
        logging.info("Connecting to database...")
        try:
            conn = connect(host=host, port=port, database=self.db)
            curs = conn.cursor()
            curs.execute(self.query)
            result = curs
            curs.close()
            conn.close()
        except Exception as e:
            logging.error("Unable to access database %s" % str(e))
        self.result_queue.append(result)

delay = 1
result_queue = []
worker1 = DatabaseWorker("db1", "select something from sometable",
        result_queue)
worker2 = DatabaseWorker("db1", "select something from othertable",
        result_queue)
worker1.start()
worker2.start()

# Wait for the job to be done
while len(result_queue) < 2:
    sleep(delay)
job_done = True
worker1.join()
worker2.join()

关于python - 如何使用python并行查询数据库,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37975904/

相关文章:

mysql - SQL 查询语法

parallel-processing - 在本书的上下文中,所谓的 "Fine-Grained Parallelism"究竟意味着什么?

r - 有没有办法跟踪并行随机森林构建过程中的进度?

c# - Parallel.ForEach finally 子句

python - 在 Geopandas map 上绘制六角网格

python - 在 Python 中定义方法别名?

python - 如何在 PyPI 项目中存储 API key ?

数据库、表和列命名约定?

python - 在 django 中设置数据库

python - 在通过 .xs : strange behavior or just me? 切片时在 multiIndex DataFrame 上使用 any() 和 all()