python - 如何使用 python 中的分页从 API 创建有效的 json 文件？

Question

我正在尝试将 Star Wars API 中的所有人员提取到一个有效的 json 文件中。

API 限制结果集，所有“人员”跨越 9 个单独的调用(即“https://swapi.co/api/people/?page=1”、“https://swapi.co/api/people/?page=2”等)。

当我循环多个请求时，我最终得到无效的 json，因为文档之间没有逗号分隔符，现在开始和结束括号。

请帮我解决我在通话中做错的事情。

import requests
import json

for x in range(1,9):  
    response = requests.get("https://swapi.co/api/people/?page="+str(x))
    data = response.json()

    next_page = data["next"] 
    results = data["results"]

    for result in results:  
        with open('data.json', 'a') as outfile:
            json.dump(result, outfile)
print('Done!')

json 文件输出:

{"name": "Luke Skywalker", "height": "172", "mass": "77", "hair_color": "blond", "skin_color": "fair", "eye_color": "blue", "birth_year": "19BBY", "gender": "male", "homeworld": "https://swapi.co/api/planets/1/", "films": ["https://swapi.co/api/films/2/", "https://swapi.co/api/films/6/", "https://swapi.co/api/films/3/", "https://swapi.co/api/films/1/", "https://swapi.co/api/films/7/"], "species": ["https://swapi.co/api/species/1/"], "vehicles": ["https://swapi.co/api/vehicles/14/", "https://swapi.co/api/vehicles/30/"], "starships": ["https://swapi.co/api/starships/12/", "https://swapi.co/api/starships/22/"], "created": "2014-12-09T13:50:51.644000Z", "edited": "2014-12-20T21:17:56.891000Z", "url": "https://swapi.co/api/people/1/"}{"name": "C-3PO", "height": "167", "mass": "75", "hair_color": "n/a", "skin_color": "gold", "eye_color": "yellow", "birth_year": "112BBY", "gender": "n/a", "homeworld": "https://swapi.co/api/planets/1/", "films": ["https://swapi.co/api/films/2/", "https://swapi.co/api/films/5/", "https://swapi.co/api/films/4/", "https://swapi.co/api/films/6/", "https://swapi.co/api/films/3/", "https://swapi.co/api/films/1/"], "species": ["https://swapi.co/api/species/2/"], "vehicles": [], "starships": [], "created": "2014-12-10T15:10:51.357000Z", "edited": "2014-12-20T21:17:50.309000Z", "url": "https://swapi.co/api/people/2/"}

Answer 1

将结果保存到内存中，然后只需执行一个 json.dump 即可解决您的问题:

import requests
import json

results = []
for x in range(1, 9):
    response = requests.get("https://swapi.co/api/people/?page="+str(x))
    data = response.json()

    next_page = data["next"]
    results.extend(data["results"])

with open('data.json', 'w') as outfile:
    json.dump(results, outfile)