python - 如何更快地使用 Google map 进行地理编码？

Question

这是我的代码，用于从 CSV 文件中的位置地址提取纬度和经度。

import pandas as pd
import requests
import json
import time

GOOGLE_MAPS_API_URL = 'https://maps.googleapis.com/maps/api/geocode/json'
API_key= 'the-key'

def gmaps_geoencoder(address):
    req = requests.get(GOOGLE_MAPS_API_URL+'?address='+address+'&key='+API_key)
    res = req.json()
    result = res['results'][0]
    lat = result['geometry']['location']['lat']
    lon = result['geometry']['location']['lng']
    return lat, lon

input_csv_file = r'path\to\location_list_100.csv'
output_csv_file = r'path\to\location_list_100_new.csv'

df = pd.read_csv(input_csv_file)

#size of chunks of data to write to the csv
chunksize = 10

t = time.time()
for i in range(len(df)):
    place = df['ADDRESS'][i]
    lat, lon, res = gmaps_geoencoder(place)
    df['Lat'][i] = lat
    df['Lon'][i] = lon

    df.to_csv(output_csv_file,
          index=False,
          chunksize=chunksize) #size of data to append for each loop

print('Time taken: '+str(time.time() - t)+'s')

100 条记录花费了 47.75818920135498s。也就是说，每条记录约 0.5 秒。我怎样才能让它更快？我有大约 100 万条记录需要转换，按照这个速度，大约需要 6 天才能完成该过程!这里花费时间的是:迭代数据帧，还是使用 gmaps API 获取数据？如果是前者，我想应该有一些方法可以让它更快。但如果是后者，有什么解决办法吗？

Answer 1

而不是那个

for i in range(len(df)):
    place = df['ADDRESS'][i]
    lat, lon, res = gmaps_geoencoder(place)
    df['Lat'][i] = lat
    df['Lon'][i] = lon

    df.to_csv(output_csv_file,
          index=False,
          chunksize=chunksize)

用这个

df[['Lat', 'Lon', 'res']] = pd.DataFrame(df['ADDRESS'].apply(lambda x: gmaps_geoencoder(x)).values.tolist())

df.to_csv(output_csv_file,
          index=False,
          chunksize=chunksize)

Refer to this link for more info