Python 到 cython - 提高大型数组迭代的性能

Question

我有以下函数，它使用生成器循环遍历大型坐标数组。由于性能确实很重要，所以我尝试将其转换为 cython。

cython 实现中是否还有其他可以提高性能的变化？也许像使用 cpython 数组或其他方式声明数组？

geometry_converter.pyx:

def esriJson_to_CV(geometry, geometry_type):
    def compress_geometry(coords):
        cdef int previous_x, previous_y, current_x, current_y
        iterator = iter(coords)
        previous_x, previous_y = iterator.next()
        yield previous_x
        yield previous_y
        for current_x, current_y in iterator:
            yield previous_x - current_x
            yield previous_y - current_y
            previous_x, previous_y = current_x, current_y

    if geometry_type == "POINT":
        converted_geometry = [int(geometry["x"]), int(geometry["y"])]
    elif geometry_type == "POLYLINE":
        converted_geometry = [list(compress_geometry(path)) for path in geometry["paths"]]
    elif geometry_type == "POLYGON":
        converted_geometry = [list(compress_geometry(ring)) for ring in geometry["rings"]]
    else:
        raise Exception("geometry_converter.esriJSON_to_CV - {} geometry type not supported".format(geometry_type))

    return converted_geometry

基准测试:

import time
from functools import wraps
import numpy as np
import geometry_converter as gc

def timethis(func):
    '''
    Decorator that reports the execution time.
    '''
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(func.__name__, end-start)
        return result
    return wrapper


def prepare_data(featCount, size):
    """create numpy array with coords and fields"""
    input = []
    for i in xrange(0, featCount):
        polygon = {"rings" : []}
        ys = np.random.uniform(0.0,89.0,size).tolist()
        xs = np.random.uniform(-179.0,179.0,size).tolist()
        polygon["rings"].append(zip(xs,ys))
        input.append(polygon)
    return input

@timethis
def process_data(data):
    output = [gc.esriJson_to_CV(x, "POLYGON") for x in data]
    return output


data = prepare_data(1000, 1000000)
out = process_data(data)
print(out[0][0][0:10])

Answer 1

Cython 并不神奇。如果不使用其静态类型，Cython 的性能提升在大多数情况下并没有真正的意义。

要获得显着的性能提升，您必须使用 cython 类型声明。

例如，不要这样做:

x = int()

你会这样做:

cdef int x

您可以在 cython documentation 中了解如何使用它们的完整说明。 .