将 csv 数据加载到 bigquery 表时,Web UI/python 客户端库等中有跳过标题行选项。 How to skip rows of csv file in BIGQUERY load API 但我在 BigQuery 的 Java 客户端库中找不到类似的选项。 目前我的代码如下
public long writeFileToTable(String datasetName, String tableName, InputStream inStream, String location)
throws IOException, InterruptedException,TimeoutException {
// [START bigquery_load_from_file]
BigQuery bigquery =BigQueryOptions.getDefaultInstance().getService();
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
.setFormatOptions(FormatOptions.csv())
.build();
// The location must be specified; other fields can be auto-detected.
JobId jobId = JobId.newBuilder().setLocation(location).build();
TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
// Write data to writer
try (OutputStream stream = Channels.newOutputStream(writer)) {
IOUtils.copy(inStream, stream);
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
// [END bigquery_load_from_file]
}
但这也在表中写入头记录。我假设 WriteChannelConfiguration 中应该提供某种方法来执行此操作。但没找到
最佳答案
抱歉,我找到了如下选项
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
.setFormatOptions(CsvOptions.newBuilder().setSkipLeadingRows(1).build())
.build();
关于java - 如何使用客户端库 (java) 在 Google BigQuery 中加载 csv 数据时跳过标题行,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/60735976/