例如,我们有大量这样的对象:
class KeyStatisticEntry:
def __init__(self, value=""):
self.usedBytes = len(value)
self.encoding = get_string_encoding(value)
@property
def total(self):
overhead = get_object_overhead(self.usedBytes)
if self.encoding == 'some value':
return overhead
else:
return self.usedBytes + overhead
@property
def aligned(self):
return some_func_with(self.usedBytes)
# Here is lots of calculated properties on basis of existing properties
而且我们需要聚合关于这个对象的大量度量——它的属性的最小值、最大值、总和、平均值、标准偏差值。目前我用这样的代码来做:
used_bytes = []
total_bytes = []
aligned_bytes = []
encodings = []
for obj in keys.items():
used_bytes.append(obj.usedBytes)
total_bytes.append(obj.total)
aligned_bytes.append(obj.aligned)
encodings.append(obj.encoding)
total_elements = len(used_bytes)
used_user = sum(used_bytes)
used_real = sum(total_bytes)
aligned = sum(aligned_bytes)
mean = statistics.mean(used_bytes)
问题:
这里有没有更“pythonic”的方式,具有更好的性能和内存使用?
最佳答案
您可以使用operator.attrgetter
来获取对象的多个属性,然后使用itertools.zip_longest
(Python 中的itertools.izip_longest
2.X ) 将相关属性附加在一起。
from operator import attrgetter
all_result = [attrgetter('usedBytes','total','aligned','encoding')(obj) for obj in keys.items()]
或者使用生成器表达式来创建生成器而不是列表:
all_result = (attrgetter('usedBytes','total','aligned','encoding')(obj) for obj in keys.items())
然后使用zip_longest
:
used_bytes, total_bytes, aligned_bytes, encodings = zip_longest(*all_results)
然后使用 map
函数将 sum
函数应用到需要求和的可迭代对象上:
used_user, used_real, aligned = map(sum,(used_bytes, total_bytes, aligned_bytes))
对于 len
和 mean
分别是:
total_elements = len(used_bytes)
mean = statistics.mean(used_bytes)
如果您想将所有子列表作为生成器处理(在内存使用方面更优化,在运行时性能方面更差),您可以使用一个新类,以便使用生成器分别计算所需结果:
from itertools import tee
class Aggregator:
def __init__(self, all_obj):
self.obj = all_obj
self.used_user, self.mean = self.getTotalBytesAndMean()
self.total_elements = len(self.all_obj)
self.aligned = self.getAligned()
def getTotalBytesAndMean(self):
iter_1, iter_2 = tee((obj.usedBytes for obj in self.all_obj))
return sum(iter_1), statistics.mean(iter_2)
def getTotal(self):
return sum(obj.total for obj in self.all_obj)
def getAligned(self):
return sum(obj.aligned for obj in self.all_obj)
def getEncoding(self):
return (obj.encoding for obj in self.all_obj)
然后你可以这样做:
Agg = Aggregator(keys.items())
# And simply access to attributes
Agg.used_user
关于以内存高效方式聚合对象属性的 Pythonic 方式?,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/35156897/