我有一些包含许多类的 python 代码。我用过 cProfile发现运行该程序的总时间为 68 秒。我在一个名为 Buyers 的类中发现了以下函数大约需要这 68 秒中的 60 秒。我必须运行该程序大约 100 次，因此任何速度的提高都会有所帮助。你能建议通过修改代码来提高速度的方法吗？如果您需要更多有用的信息，请告诉我。
def qtyDemanded(self, timePd, priceVector):
'''Returns quantity demanded in period timePd. In addition,
also updates the list of customers and non-customers.
Inputs: timePd and priceVector
Output: count of people for whom priceVector[-1] < utility
'''
## Initialize count of customers to zero
## Set self.customers and self.nonCustomers to empty lists
price = priceVector[-1]
count = 0
self.customers = []
self.nonCustomers = []
for person in self.people:
if person.utility >= price:
person.customer = 1
self.customers.append(person)
else:
person.customer = 0
self.nonCustomers.append(person)
return len(self.customers)
self.people是 person 的列表对象。每个person有 customer和 utility作为它的属性。编辑 - 回复添加
-------------------------------------
非常感谢您的建议。这里是
回应人们善意的一些问题和建议
制作。我还没有全部尝试过，但会尝试其他的，稍后再写。
(1) @amber - 函数被访问 80,000 次。
(2) @gnibbler 和其他人 - self.people 是内存中的 Person 对象列表。未连接到数据库。
(3) @Hugh Bothwell
原始函数占用的 cumtime - 60.8 s(访问 80000 次)
新函数采用本地函数别名作为建议的 cumtime - 56.4 s(访问 80000 次)
(4) @rotoglup 和 @Martin Thomas
我还没有尝试过你的解决方案。我需要检查其余代码以查看我使用 self.customers 的地方，然后才能更改不将客户附加到 self.customers 列表。但我会尝试这个并回信。
(5) @TryPyPy - 感谢您提供检查代码的好意。
让我首先阅读一下您提出的建议，看看这些建议是否可行。
编辑 2
有人建议，因为我在
self.people 中标记了客户和非客户。 ，我应该尝试不创建 self.customers 的单独列表和 self.noncustomers使用附加。相反，我应该遍历 self.people查找客户数量。我尝试了以下代码并对以下两个函数进行计时 f_w_append和 f_wo_append .我确实发现后者花费的时间更少，但仍然是前者花费的时间的 96%。也就是说，速度的增加非常小。@TryPyPy - 以下代码足以检查瓶颈函数，以防您的报价仍然存在以与其他编译器进行检查。
再次感谢所有回复的人。
import numpy
class person(object):
def __init__(self, util):
self.utility = util
self.customer = 0
class population(object):
def __init__(self, numpeople):
self.people = []
self.cus = []
self.noncus = []
numpy.random.seed(1)
utils = numpy.random.uniform(0, 300, numpeople)
for u in utils:
per = person(u)
self.people.append(per)
popn = population(300)
def f_w_append():
'''Function with append'''
P = 75
cus = []
noncus = []
for per in popn.people:
if per.utility >= P:
per.customer = 1
cus.append(per)
else:
per.customer = 0
noncus.append(per)
return len(cus)
def f_wo_append():
'''Function without append'''
P = 75
for per in popn.people:
if per.utility >= P:
per.customer = 1
else:
per.customer = 0
numcustomers = 0
for per in popn.people:
if per.customer == 1:
numcustomers += 1
return numcustomers
编辑 3:似乎 numpy 是问题
这是对约翰·马钦 (John Machin) 下面所说的话的回应。下面是定义
Population 的两种方式类(class)。我运行了下面的程序两次，每次创建 Population类(class)。一种使用 numpy，一种不使用 numpy。一个没有 numpy 花费的时间与约翰在运行中发现的时间相似。一个 numpy 需要更长的时间。我不清楚的是 popn实例是在时间记录开始之前创建的(至少它是从代码中显示的)。那么，为什么 numpy 版本需要更长的时间。而且，我认为 numpy 应该更有效率。无论如何，问题似乎出在 numpy 而不是 append，即使它确实减慢了一些速度。有人可以用下面的代码确认吗？谢谢。import random # instead of numpy
import numpy
import time
timer_func = time.time # using Mac OS X 10.5.8
class Person(object):
def __init__(self, util):
self.utility = util
self.customer = 0
class Population(object):
def __init__(self, numpeople):
random.seed(1)
self.people = [Person(random.uniform(0, 300)) for i in xrange(numpeople)]
self.cus = []
self.noncus = []
# Numpy based
# class Population(object):
# def __init__(self, numpeople):
# numpy.random.seed(1)
# utils = numpy.random.uniform(0, 300, numpeople)
# self.people = [Person(u) for u in utils]
# self.cus = []
# self.noncus = []
def f_wo_append(popn):
'''Function without append'''
P = 75
for per in popn.people:
if per.utility >= P:
per.customer = 1
else:
per.customer = 0
numcustomers = 0
for per in popn.people:
if per.customer == 1:
numcustomers += 1
return numcustomers
t0 = timer_func()
for i in xrange(20000):
x = f_wo_append(popn)
t1 = timer_func()
print t1-t0
编辑 4:查看 John Machin 和 TryPyPy 的答案
由于这里的编辑和更新太多了，第一次来到这里的人可能会有些困惑。请参阅 John Machin 和 TryPyPy 的答案。这两者都可以帮助大大提高代码的速度。我很感谢他们和其他提醒我
append 缓慢的人.由于在这种情况下我将使用 John Machin 的解决方案而不是使用 numpy 来生成实用程序，因此我接受他的回答作为答案。但是，我也非常感谢 TryPyPy 指出的方向。
最佳答案
在优化 Python 代码以提高速度后，您可以尝试许多事情。如果这个程序不需要C扩展，你可以在PyPy下运行它受益于其 JIT 编译器。您可以尝试制作 C extension对于可能 huge speedups . Shed Skin甚至允许您将 Python 程序转换为独立的 C++ 二进制文件。
如果您能为基准测试提供足够的代码，我愿意在这些不同的优化方案下对您的程序进行计时，
编辑 : 首先，我必须同意其他人的看法:你确定你测量的时间正确吗？示例代码在此处的 0.1 秒内运行了 100 次，因此很有可能时间错误或您遇到了代码示例中不存在的瓶颈(IO？)。
也就是说，我做了 300000 人，所以时间是一致的。这是 CPython (2.5)、PyPy 和 Shed Skin 共享的改编代码:
from time import time
import random
import sys
class person(object):
def __init__(self, util):
self.utility = util
self.customer = 0
class population(object):
def __init__(self, numpeople, util):
self.people = []
self.cus = []
self.noncus = []
for u in util:
per = person(u)
self.people.append(per)
def f_w_append(popn):
'''Function with append'''
P = 75
cus = []
noncus = []
# Help CPython a bit
# cus_append, noncus_append = cus.append, noncus.append
for per in popn.people:
if per.utility >= P:
per.customer = 1
cus.append(per)
else:
per.customer = 0
noncus.append(per)
return len(cus)
def f_wo_append(popn):
'''Function without append'''
P = 75
for per in popn.people:
if per.utility >= P:
per.customer = 1
else:
per.customer = 0
numcustomers = 0
for per in popn.people:
if per.customer == 1:
numcustomers += 1
return numcustomers
def main():
try:
numpeople = int(sys.argv[1])
except:
numpeople = 300000
print "Running for %s people, 100 times." % numpeople
begin = time()
random.seed(1)
# Help CPython a bit
uniform = random.uniform
util = [uniform(0.0, 300.0) for _ in xrange(numpeople)]
# util = [random.uniform(0.0, 300.0) for _ in xrange(numpeople)]
popn1 = population(numpeople, util)
start = time()
for _ in xrange(100):
r = f_wo_append(popn1)
print r
print "Without append: %s" % (time() - start)
popn2 = population(numpeople, util)
start = time()
for _ in xrange(100):
r = f_w_append(popn2)
print r
print "With append: %s" % (time() - start)
print "\n\nTotal time: %s" % (time() - begin)
if __name__ == "__main__":
main()
使用 PyPy 运行就像使用 CPython 运行一样简单，您只需键入 'pypy' 而不是 'python'。对于 Shed Skin，您必须转换为 C++，编译并运行:
shedskin -e makefaster.py && make
# Check that you're using the makefaster.so file and run test
python -c "import makefaster; print makefaster.__file__; makefaster.main()"
这是 Cython 化的代码:
from time import time
import random
import sys
cdef class person:
cdef readonly int utility
cdef public int customer
def __init__(self, util):
self.utility = util
self.customer = 0
class population(object):
def __init__(self, numpeople, util):
self.people = []
self.cus = []
self.noncus = []
for u in util:
per = person(u)
self.people.append(per)
cdef int f_w_append(popn):
'''Function with append'''
cdef int P = 75
cdef person per
cus = []
noncus = []
# Help CPython a bit
# cus_append, noncus_append = cus.append, noncus.append
for per in popn.people:
if per.utility >= P:
per.customer = 1
cus.append(per)
else:
per.customer = 0
noncus.append(per)
cdef int lcus = len(cus)
return lcus
cdef int f_wo_append(popn):
'''Function without append'''
cdef int P = 75
cdef person per
for per in popn.people:
if per.utility >= P:
per.customer = 1
else:
per.customer = 0
cdef int numcustomers = 0
for per in popn.people:
if per.customer == 1:
numcustomers += 1
return numcustomers
def main():
cdef int i, r, numpeople
cdef double _0, _300
_0 = 0.0
_300 = 300.0
try:
numpeople = int(sys.argv[1])
except:
numpeople = 300000
print "Running for %s people, 100 times." % numpeople
begin = time()
random.seed(1)
# Help CPython a bit
uniform = random.uniform
util = [uniform(_0, _300) for i in xrange(numpeople)]
# util = [random.uniform(0.0, 300.0) for _ in xrange(numpeople)]
popn1 = population(numpeople, util)
start = time()
for i in xrange(100):
r = f_wo_append(popn1)
print r
print "Without append: %s" % (time() - start)
popn2 = population(numpeople, util)
start = time()
for i in xrange(100):
r = f_w_append(popn2)
print r
print "With append: %s" % (time() - start)
print "\n\nTotal time: %s" % (time() - begin)
if __name__ == "__main__":
main()
为了构建它，有一个像这样的 setup.py 很好:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [Extension("cymakefaster", ["makefaster.pyx"])]
setup(
name = 'Python code to speed up',
cmdclass = {'build_ext': build_ext},
ext_modules = ext_modules
)
你用以下方法构建它:
python setupfaster.py build_ext --inplace
然后测试:
python -c“导入cymakefaster；打印cymakefaster。文件；cymakefaster.main()”
每个版本的计时运行五次，Cython 是最快和最容易使用的代码生成器(Shed Skin 的目标是更简单，但神秘的错误消息和隐式静态类型在这里变得更难)。至于最佳值(value)，PyPy 在没有代码更改的计数器版本中提供了令人印象深刻的加速。
#Results (time in seconds for 30000 people, 100 calls for each function):
Mean Min Times
CPython 2.5.2
Without append: 35.037 34.518 35.124, 36.363, 34.518, 34.620, 34.559
With append: 29.251 29.126 29.339, 29.257, 29.259, 29.126, 29.272
Total time: 69.288 68.739 69.519, 70.614, 68.746, 68.739, 68.823
PyPy 1.4.1
Without append: 2.672 2.655 2.655, 2.670, 2.676, 2.690, 2.668
With append: 13.030 12.672 12.680, 12.725, 14.319, 12.755, 12.672
Total time: 16.551 16.194 16.196, 16.229, 17.840, 16.295, 16.194
Shed Skin 0.7 (gcc -O2)
Without append: 1.601 1.599 1.599, 1.605, 1.600, 1.602, 1.599
With append: 3.811 3.786 3.839, 3.795, 3.798, 3.786, 3.839
Total time: 5.704 5.677 5.715, 5.705, 5.699, 5.677, 5.726
Cython 0.14 (gcc -O2)
Without append: 1.692 1.673 1.673, 1.710, 1.678, 1.688, 1.711
With append: 3.087 3.067 3.079, 3.080, 3.119, 3.090, 3.067
Total time: 5.565 5.561 5.562, 5.561, 5.567, 5.562, 5.572
编辑 :Aaa 和更有意义的时间，对于 80000 个调用，每个调用 300 人:
Results (time in seconds for 300 people, 80000 calls for each function):
Mean Min Times
CPython 2.5.2
Without append: 27.790 25.827 25.827, 27.315, 27.985, 28.211, 29.612
With append: 26.449 24.721 24.721, 27.017, 27.653, 25.576, 27.277
Total time: 54.243 50.550 50.550, 54.334, 55.652, 53.789, 56.892
Cython 0.14 (gcc -O2)
Without append: 1.819 1.760 1.760, 1.794, 1.843, 1.827, 1.871
With append: 2.089 2.063 2.100, 2.063, 2.098, 2.104, 2.078
Total time: 3.910 3.859 3.865, 3.859, 3.944, 3.934, 3.951
PyPy 1.4.1
Without append: 0.889 0.887 0.894, 0.888, 0.890, 0.888, 0.887
With append: 1.671 1.665 1.665, 1.666, 1.671, 1.673, 1.681
Total time: 2.561 2.555 2.560, 2.555, 2.561, 2.561, 2.569
Shed Skin 0.7 (g++ -O2)
Without append: 0.310 0.301 0.301, 0.308, 0.317, 0.320, 0.303
With append: 1.712 1.690 1.733, 1.700, 1.735, 1.690, 1.702
Total time: 2.027 2.008 2.035, 2.008, 2.052, 2.011, 2.029
Shed Skin 变得最快，PyPy 超过 Cython。与 CPython 相比，这三个都大大加快了速度。
关于python - 提高python代码的速度，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/4653715/