javascript - 如何流式传输 OpenAI 的完成 API？

Question

我想流式传输 completion via OpenAI's API 的结果.

文档提到使用 server-sent events - 好像是这样isn't handled out of the box for flask所以我试图在客户端进行(我知道这会暴露 API key )。然而，因为 OpenAI API 需要它发布它看起来像 isn't compatible with the eventSource API .我尝试通过获取 (Using readable streams) 进行操作，但是当我尝试通过示例转换为 JSON 时，出现以下错误:Uncaught (in promise) SyntaxError: Unexpected token 'd', "data: {"id"... is not valid JSON(我知道这不是有效的 JSON)。它似乎在解析整个结果，而不是每个单独的流。

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": " to", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": " AL", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": "I", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

会喜欢一些指示或一个简单的代码示例来说明如何执行此操作，因为我已经苦思冥想了一段时间。谢谢!

Answer 1

终于得到了这个工作代码:

import { Configuration, OpenAIApi } from "openai";
import dotenv from "dotenv";
dotenv.config({ override: true });

const openai = new OpenAIApi(new Configuration({ apiKey: process.env.OPENAI_KEY }));

const getText = async (prompt, callback) => {
    const completion = await openai.createCompletion(
        {
            model: "text-davinci-003",
            prompt: prompt,
            max_tokens: 1000,
            stream: true,
        },
        { responseType: "stream" }
    );
    return new Promise((resolve) => {
        let result = "";
        completion.data.on("data", (data) => {
            const lines = data
                ?.toString()
                ?.split("\n")
                .filter((line) => line.trim() !== "");
            for (const line of lines) {
                const message = line.replace(/^data: /, "");
                if (message == "[DONE]") {
                    resolve(result);
                } else {
                    let token;
                    try {
                        token = JSON.parse(message)?.choices?.[0]?.text;
                    } catch {
                        console.log("ERROR", json);
                    }
                    result += token;
                    if (token) {
                        callback(token);
                    }
                }
            }
        });
    });
};
    
console.log(await getText("Who was the latest president of USA?", (c) => process.stdout.write(c)));