我们有一个应用程序，到目前为止一直只保留在前端，但是当使用大型网格配置对其进行测试时，我们遇到了一些处理和内存限制。因此，我们正在探索将资源要求较高的任务推送到后端的选项。
因此，我目前正在运行一些基准测试，以获取我们可能看到的性能差异的一些迹象。更具体地说，考虑到性能瓶颈的很大一部分是在嵌套循环上，我用 js 和 python 编写了一个简单的代码片段来测试处理嵌套循环和构建不同大小的数组的性能。
令我惊讶的是，无论我测试过什么排列，js 似乎始终更快。
JS 片段:
var timeTotal = 0;
var benchmarkTests = 100;
var testSizes = [50, 100, 500, 1000, 3000];
for (var a = 0; a < testSizes.length; a++) {
var minX = 0;
var maxX = testSizes[a];
var minY = 0;
var maxY = testSizes[a];
var cellDimensionsX = 0.991; // cell width
var cellDimensionsY = 1.652; // cell height
var cellDimensionsXHalf = cellDimensionsX / 2; // half cell width
var cellDimensionsYHalf = cellDimensionsY / 2; // half cell height
var maxCellsXCount = Math.floor(maxX / cellDimensionsX);
var maxCellsYCount = Math.floor(maxY / cellDimensionsY);
console.log("X:", maxCellsXCount, "| Y", maxCellsYCount, "| Total", maxCellsXCount * maxCellsYCount);
for (var k = 0; k < benchmarkTests; k++) {
var cellCoordsTime = new Date().getTime();
var cellCoords = {};
var index = 0;
for (var i = 0; i < maxCellsXCount; i++) {
var xCoord = (i * cellDimensionsX) + cellDimensionsXHalf;
for (var d = 0; d < maxCellsYCount; d++) {
cellCoords[index] = {
x: xCoord,
y: (d * cellDimensionsY) + cellDimensionsYHalf
};
index++;
}
}
var thisTime = new Date().getTime() - cellCoordsTime;
timeTotal += thisTime;
// console.log('cellCoords', thisTime, 'for grid with', index, 'cells');
}
console.log('Testing with a', testSizes[a], '*', testSizes[a], 'grid area. Total time', timeTotal, '. Avg', (timeTotal / benchmarkTests), 'for', benchmarkTests, 'tests');
}
Python 代码片段:
#!/usr/bin/python
import time
timeTotal = 0
benchmarkTests = 100
testSizes = [50, 100, 500, 1000, 3000]
for a in range(len(testSizes)):
minX = 0
maxX = testSizes[a]
minY = 0
maxY = testSizes[a]
cellDimensionsX = 0.991 # cell width
cellDimensionsY = 1.652 # cell height
cellDimensionsXHalf = cellDimensionsX / 2 # half cell width
cellDimensionsYHalf = cellDimensionsY / 2 # half cell height
maxCellsXCount = int(maxX / cellDimensionsX)
maxCellsYCount = int(maxY / cellDimensionsY)
print("X: %s | Y %s | Total %s" % (maxCellsXCount, maxCellsYCount, maxCellsXCount * maxCellsYCount))
for k in range(benchmarkTests):
start = time.time()
cellCoords = {}
index = 0
for i in range(maxCellsXCount):
xCoord = (i * cellDimensionsX) + cellDimensionsXHalf
for d in range(maxCellsYCount):
cellCoords[index] = {'x': xCoord, 'y': (d * cellDimensionsY) + cellDimensionsYHalf}
index += 1
thisTime = (time.time() - start) * 1000;
timeTotal = timeTotal + thisTime;
# print("Elapsed Time: %s for grid with %s cells" % (thisTime, index))
print("Testing with a %s*%s grid area. Total time %s. Avg %s for %s tests" % (testSizes[a], testSizes[a], timeTotal, (timeTotal / benchmarkTests), benchmarkTests))
运行这些时我得到:
JS:
Testing with a 50 * 50 grid area. Total time 26 . Avg 0.26 for 100 tests
Testing with a 100 * 100 grid area. Total time 85 . Avg 0.85 for 100 tests
Testing with a 500 * 500 grid area. Total time 4539 . Avg 45.39 for 100 tests
Testing with a 1000 * 1000 grid area. Total time 23160 . Avg 231.6 for 100 tests
Testing with a 3000 * 3000 grid area. Total time 243760 . Avg 2437.6 for 100 tests
Python:
Testing with a 50*50 grid area. Total time 50.8642196655. Avg 0.508642196655 for 100 tests
Testing with a 100*100 grid area. Total time 262.931108475. Avg 2.62931108475 for 100 tests
Testing with a 500*500 grid area. Total time 6338.83333206. Avg 63.3883333206 for 100 tests
Testing with a 1000*1000 grid area. Total time 30769.4478035. Avg 307.694478035 for 100 tests
Testing with a 3000*3000 grid area. Total time 304995.391846. Avg 3049.95391846 for 100 tests
所有时间均以毫秒为单位，并且所有测试都在同一本地主机上运行。
我本来期望 python 比 js 快得多。我有什么遗漏的吗？
更新#1(将所有内容移至函数中 | 总体增益约 17%):
Testing with a 50*50 grid area. Total time 41.2473678589. Avg 0.412473678589 for 100 tests
Testing with a 100*100 grid area. Total time 174.555540085. Avg 1.74555540085 for 100 tests
Testing with a 500*500 grid area. Total time 5617.09475517. Avg 56.1709475517 for 100 tests
Testing with a 1000*1000 grid area. Total time 21199.390173. Avg 211.99390173 for 100 tests
Testing with a 3000*3000 grid area. Total time 255921.251535. Avg 2559.21251535 for 100 tests
更新 #2(将范围交换为 xrange | 更新 #1 后总体额外增益 ~15% | 与初始代码相比总体增益 ~30%):
Testing with a 50*50 grid area. Total time 38.7289524078. Avg 0.387289524078 for 100 tests
Testing with a 100*100 grid area. Total time 176.453590393. Avg 1.76453590393 for 100 tests
Testing with a 500*500 grid area. Total time 5346.49443626. Avg 53.4649443626 for 100 tests
Testing with a 1000*1000 grid area. Total time 21618.1008816. Avg 216.181008816 for 100 tests
Testing with a 3000*3000 grid area. Total time 213622.769356. Avg 2136.22769356 for 100 tests
更新 #3(将字典交换为列表 | 更新 #2 后总体额外增益 ~35% | 与初始代码相比总体增益 ~55%):
Testing with a 50*50 grid area. Total time 20.7185745239. Avg 0.207185745239 for 100 tests
Testing with a 100*100 grid area. Total time 100.9953022. Avg 1.009953022 for 100 tests
Testing with a 500*500 grid area. Total time 3033.61153603. Avg 30.3361153603 for 100 tests
Testing with a 1000*1000 grid area. Total time 12399.708271. Avg 123.99708271 for 100 tests
Testing with a 3000*3000 grid area. Total time 140118.921518. Avg 1401.18921518 for 100 tests
将 JS 与 Python 的 UPDATE #3 相匹配(将对象交换为数组 | 与初始代码相比总体损失约 165%):
Testing with a 50 * 50 grid area. Total time 30 . Avg 0.3 for 100 tests
Testing with a 100 * 100 grid area. Total time 48 . Avg 0.48 for 100 tests
Testing with a 500 * 500 grid area. Total time 12694 . Avg 126.94 for 100 tests
Testing with a 1000 * 1000 grid area. Total time 81402 . Avg 814.02 for 100 tests
Testing with a 3000 * 3000 grid area. Total time 625615 . Avg 6256.15 for 100 tests
更新 #4(将 Cython 拉入战斗 | 更新 #3 后总体额外增益 ~26% | 与初始代码相比总体增益 ~66%):
Testing with a 50*50 grid area. Total time 30. Avg 0.300 for 100 tests
Testing with a 100*100 grid area. Total time 68. Avg 0.680 for 100 tests
Testing with a 500*500 grid area. Total time 2475. Avg 24.750 for 100 tests
Testing with a 1000*1000 grid area. Total time 9924. Avg 99.240 for 100 tests
Testing with a 3000*3000 grid area. Total time 101697. Avg 1016.970 for 100 tests
更新 #5(输入变量 | 更新 #4 后总体额外增益 ~23% | 与初始代码相比总体增益 ~74%):
Testing with a 50*50 grid area. Total time 5. Avg 0.048 for 100 tests
Testing with a 100*100 grid area. Total time 43. Avg 0.426 for 100 tests
Testing with a 500*500 grid area. Total time 1851. Avg 18.511 for 100 tests
Testing with a 1000*1000 grid area. Total time 8020. Avg 80.202 for 100 tests
Testing with a 3000*3000 grid area. Total time 78350. Avg 783.502 for 100 tests
(希望最终)更新 #6(Cython 相关优化，包括在 2 个核心上并行化 | 更新 #5 后总体额外增益 ~88% | 与初始代码相比总体增益 ~97%):
Testing with a 50*50 grid area. Total time 0.668. Avg 0.007 for 100 tests
Testing with a 100*100 grid area. Total time 1.584. Avg 0.016 for 100 tests
Testing with a 500*500 grid area. Total time 57.374. Avg 0.574 for 100 tests
Testing with a 1000*1000 grid area. Total time 521.210. Avg 5.212 for 100 tests
Testing with a 3000*3000 grid area. Total time 10113.633. Avg 101.136 for 100 tests
与原始实现相比，当前版本的平均速度快了 30 到 35 倍。所以暂时告辞了。
最终更新(在超线程 8 核机器上测试(不再是本地主机，不再进行同类比较)| 进一步与 cython 相关的优化，并缓存 y 的值 | 更新 #6 后总体额外增益 ~78% | 与初始代码相比总体增益 ~99.3%):
Testing with a 50*50 grid area. Total time 0.498. Avg 0.005 for 100 tests
Testing with a 100*100 grid area. Total time 1.146. Avg 0.011 for 100 tests
Testing with a 500*500 grid area. Total time 22.856. Avg 0.229 for 100 tests
Testing with a 1000*1000 grid area. Total time 113.819. Avg 1.138 for 100 tests
Testing with a 3000*3000 grid area. Total time 2228.098ms. Avg 22.281 for 100 tests
Testing with a 10000*10000 grid area. Total time 29407.874ms. Avg 294.79 for 100 tests
Testing with a 20000*20000 grid area. Total time 157185.469ms. Avg 1571.855 for 100 tests
最佳答案
使用更适合 Python 的数据类型(元组而不是创建 {x: .., y: ..} 字典，使用列表而不是具有连续整数索引的字典)，使用xrange 而不是 range，并将所有内容包装到一个函数中，给出以下代码:
import time
def foo():
timeTotal = 0
benchmarkTests = 100
testSizes = [50, 100, 500, 1000, 3000]
for a in range(len(testSizes)):
minX = 0
maxX = testSizes[a]
minY = 0
maxY = testSizes[a]
cellDimensionsX = 0.991 # cell width
cellDimensionsY = 1.652 # cell height
cellDimensionsXHalf = cellDimensionsX / 2 # half cell width
cellDimensionsYHalf = cellDimensionsY / 2 # half cell height
maxCellsXCount = int(maxX / cellDimensionsX)
maxCellsYCount = int(maxY / cellDimensionsY)
print("X: %s | Y %s | Total %s" % (maxCellsXCount, maxCellsYCount, maxCellsXCount * maxCellsYCount))
for k in xrange(benchmarkTests):
start = time.time()
# cellCoords = {}
cellCoords = []
for i in xrange(maxCellsXCount):
xCoord = (i * cellDimensionsX) + cellDimensionsXHalf
for d in xrange(maxCellsYCount):
# cellCoords[index] = {'x': xCoord, 'y': (d * cellDimensionsY) + cellDimensionsYHalf}
# cellCoords[index] = (xCoord, (d * cellDimensionsY) + cellDimensionsYHalf)
cellCoords.append((xCoord, (d * cellDimensionsY) + cellDimensionsYHalf))
# index += 1
thisTime = (time.time() - start) * 1000;
timeTotal = timeTotal + thisTime;
# print("Elapsed Time: %s for grid with %s cells" % (thisTime, index))
print("Testing with a %s*%s grid area. Total time %s. Avg %s for %s tests" % (testSizes[a], testSizes[a], timeTotal, (timeTotal / benchmarkTests), benchmarkTests))
foo()
提供大约 50% 的加速:
X: 50 | Y 30 | Total 1500
Testing with a 50*50 grid area. Total time 33.9999198914. Avg 0.339999198914 for 100 tests
X: 100 | Y 60 | Total 6000
Testing with a 100*100 grid area. Total time 191.999912262. Avg 1.91999912262 for 100 tests
X: 504 | Y 302 | Total 152208
Testing with a 500*500 grid area. Total time 4790.99988937. Avg 47.9099988937 for 100 tests
X: 1009 | Y 605 | Total 610445
Testing with a 1000*1000 grid area. Total time 24529.9999714. Avg 245.299999714 for 100 tests
X: 3027 | Y 1815 | Total 5494005
Testing with a 3000*3000 grid area. Total time 201085.000038. Avg 2010.85000038 for 100 tests
与我机器上的原始代码相比:
X: 50 | Y 30 | Total 1500
Testing with a 50*50 grid area. Total time 94.0001010895. Avg 0.940001010895 for 100 tests
X: 100 | Y 60 | Total 6000
Testing with a 100*100 grid area. Total time 495.000123978. Avg 4.95000123978 for 100 tests
X: 504 | Y 302 | Total 152208
Testing with a 500*500 grid area. Total time 10732.0001125. Avg 107.320001125 for 100 tests
X: 1009 | Y 605 | Total 610445
Testing with a 1000*1000 grid area. Total time 49074.0001202. Avg 490.740001202 for 100 tests
X: 3027 | Y 1815 | Total 5494005
Traceback (most recent call last):
File "pyperf-orig.py", line 31, in <module>
cellCoords[index] = {'x': xCoord, 'y': (d * cellDimensionsY) + cellDimensionsYHalf}
MemoryError
关于javascript - JS 和 Python 之间的基准测试性能差异，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/54932698/