在JPA中,有没有办法批量将数据插入数据库,并且仅在数据库不存在时才插入。当我尝试批量插入时,由于唯一键约束,它会抛出异常。我想插入数据库中不存在的所有记录,其余的应该跳过。
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [reference_number_master_id_key]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
最佳答案
我和我的团队最终创建了 JpaRepositories 扩展的实现来添加此类行为。
主界面
public interface BatchInsertable <T> {
enum Mode {NORMAL, IGNORE, REPLACE}
void batchInsertUsingMode(Collection<T> items, final Mode mode);
}
这就是我们将其连接到现有 JpaRepository 的方式。
public interface UrlRepository extends JpaRepository<UrlEntity, Long>, BatchInsertable<UrlEntity> {
// insert common methods
}
...及其用法
urlRepository.batchInsertUsingMode(newUrlEntities, BatchInsertable.Mode.IGNORE);
批量插入器实现
@Component
@Scope("prototype")
@Lazy
public class BatchInserter<T> extends AbstractBaseBatchJdbcMutation<T> implements BatchInsertable<T> {
@Override
public void batchInsertUsingMode(final Collection<T> items, final Mode mode) {
final Map<String, String> parameters = new HashMap<>();
parameters.put("MODE", mode == Mode.IGNORE ? "INSERT IGNORE" : mode == Mode.REPLACE ? "REPLACE" : "INSERT");
parameters.put("STAGING_TABLE", getTableName());
parameters.put("COLUMNS", buildColumnNameList());
parameters.put("PLACEHOLDERS", buildBindVariablePlaceholderList());
final StrSubstitutor strSubstitutor = new StrSubstitutor(parameters);
final String sqlTemplate = "${MODE} INTO `${STAGING_TABLE}` (${COLUMNS}) VALUES (${PLACEHOLDERS})";
getJdbcTemplate().batchUpdate(strSubstitutor.replace(sqlTemplate), buildBindVariables(items));
}
}
批量删除器实现
@Component
@Scope("prototype")
@Lazy
public class BatchDeleter<T> extends AbstractBaseBatchJdbcMutation<T> implements BatchDeletable<T> {
@Override
public int batchDelete(final List<T> items) {
final Map<String, String> parameters = new HashMap<>();
parameters.put("STAGING_TABLE", getTableName());
parameters.put("COLUMNS", buildColumnNameList());
parameters.put("PLACEHOLDERS", buildBindVariablePlaceholderList());
final StrSubstitutor strSubstitutor = new StrSubstitutor(parameters);
final String sqlTemplate = "DELETE FROM `${STAGING_TABLE}` WHERE (${COLUMNS}) = (${PLACEHOLDERS})";
final int[] updateCounts = getJdbcTemplate().batchUpdate(strSubstitutor.replace(sqlTemplate), buildBindVariables(items));
return sum(updateCounts);
}
private static int sum(final int[] updateCounts) {
int sum = 0;
for (final int updateCount : updateCounts) {
sum += updateCount;
}
return sum;
}
}
维护通用配置绑定(bind)逻辑的抽象类
public abstract class AbstractBaseBatchJdbcMutation<T> {
private JdbcTemplate jdbcTemplate;
private List<ColumnValueExtractors> columnExtractors;
private String tableName;
public void setColumnExtractors(final List<ColumnValueExtractors> columnExtractors) {
this.columnExtractors = new ArrayList<>(columnExtractors);
}
public void setTableName(final String tableName) {
this.tableName = tableName;
}
protected List<Object[]> buildBindVariables(final Collection<T> items) {
return FluentIterable.from(items).transform(new BulkBindingTransform<T>(columnExtractors)).toList();
}
protected JdbcTemplate getJdbcTemplate() {
return jdbcTemplate;
}
protected String getTableName() {
return tableName;
}
@Autowired
public void setDataSource(final DataSource datasource) {
this.jdbcTemplate = new JdbcTemplate(datasource);
}
protected String buildColumnNameList() {
return join(extract(columnExtractors, on(ColumnValueExtractors.class).getColumnName()), ",");
}
protected String buildBindVariablePlaceholderList() {
return join(nCopies(columnExtractors.size(), "?"), ",");
}
private static class BulkBindingTransform<T> implements Function<T, Object[]> {
private final List<ColumnValueExtractors> columns;
private BulkBindingTransform(final List<ColumnValueExtractors> columns) {
this.columns = columns;
}
@Nullable
@Override
public Object[] apply(final T input) {
final Object[] bindings = new Object[columns.size()];
for (int i = 0; i < columns.size(); i++) {
bindings[i] = columns.get(i).resolveValue(input);
}
return bindings;
}
}
}
这还可以让您绕过直接与默认 save(Iterable<S> iterable)
连接时可能遇到的一些缓慢问题。界面。我们将它全部用于批量 SQL 操作。令人惊讶的是,如此简单的任务竟然如此复杂。我敢打赌,您可以缩减此值以满足您的特定需求。 :)
关于java - 在JPA中,有没有办法批量将数据插入数据库,并且仅在数据库中不存在时插入,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37476592/