java - 通过 Java Web 应用程序从查询中下载非常大的结果

Question

最近一直在思考一个问题，不知道大家有没有什么建议。 我有一个 Web 应用程序，需要添加一个用于报告的部分。它将根据用户 gui 输入构建一个 t-sql 查询，执行并允许用户通过浏览器将文件作为 csv 下载到他们的计算机上。就其本身而言，这些都不是太复杂。我以前做过这些事情。然而，有一个问题我以前没有解决过。结果可能最终是值(value)多个 GB 的数据。我会采取哪些与正常情况不同的做法(就代码而言)以防止浏览器过载或速度极慢？显然不能一次性全部保存在内存中吗？我该如何防止这种情况发生？

Answer 1

您可以将 csv 数据直接写入传入请求的 OutputStream:

package example;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import java.io.IOException;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.nio.charset.StandardCharsets;

public class MyServlet extends HttpServlet {

    private final CSVCreator csvCreator;

    private MyServlet(CSVCreator csvCreator) {
        this.csvCreator = csvCreator;
    }

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        // be nice to your consumer, tell them what they get
        resp.setHeader("Content-Type", "text/comma-separated-values; charset=UTF-8");

        try (Writer writer = new OutputStreamWriter(resp.getOutputStream(), StandardCharsets.UTF_8)) {
            this.csvCreator.createCSV("some params you need to create the correct csv", writer);
        }
    }
}

您必须更改 csv 生成才能写入写入器...例如:

package example;

import java.io.IOException;

public class CSVCreator {

    public void createCSV(String someParamsYouNeedToCreateTheCSV, Appendable appendable) throws IOException {
        appendable.append("Column1,Column2,Column3\n");
        appendable.append("Row1,Row1,Row1\n");
        appendable.append("Row2,Row2,Row2\n");
    }
}

例如，您还可以使用 apache-commons-csv 将 csv 写入给定的 Appendable(请参阅: https://javadoc.io/doc/org.apache.commons/commons-csv/latest/org/apache/commons/csv/CSVPrinter.html )

package example;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;

import java.io.IOException;

public class CSVCreator {

    public void createCSV(String someParamsYouNeedToCreateTheCSV, Appendable appendable) throws IOException {
        try (CSVPrinter printer = new CSVPrinter(appendable, CSVFormat.DEFAULT)) {
            printer.printRecord("Column1", "Column2", "Column3");
            printer.printRecord("Row1", "Row1", "Row1");
            printer.printRecord("Row2", "Row2", "Row2");
        }
    }
}

编辑:

您可以从结果集中创建 CSV 文件，如下所示:

package example;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;

import java.io.IOException;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;

public class ResultSetToCSV {

    public void createCSV(ResultSet rs, Appendable appendable) throws IOException, SQLException {
        final ResultSetMetaData metaData = rs.getMetaData();
        final int columnCount = metaData.getColumnCount();

        try (CSVPrinter printer = new CSVPrinter(appendable, CSVFormat.DEFAULT)) {
            printHeaders(metaData, columnCount, printer);
            printValues(rs, columnCount, printer);
        }
    }

    private void printHeaders(ResultSetMetaData metaData, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        for (int i = 1; i <= columnCount; i++) {
            printer.print(metaData.getColumnLabel(i));
        }

        printer.println();
    }

    private void printValues(ResultSet rs, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        while (rs.next()) {
            printValue(rs, columnCount, printer);
        }
    }

    private void printValue(ResultSet rs, int columnCount, CSVPrinter printer) throws IOException, SQLException {
        for (int i = 1; i <= columnCount; i++) {
            printer.print(rs.getString(i));
        }

        printer.println();
    }
}