java - 批量保存的 Spring Data JPA 性能问题

Question

我有以下实体

@Entity
@Table(name = "APP_ITEM")
public class AppItem implements Serializable {
private static final long serialVersionUID = 1L;

@EmbeddedId
private AppItemPK AppItemPK;

 public AppItemPK getAppItemPK() {
    return appItemPK;
}

public void setAppItemPK(
        AppItemPK appItemPK) {
    this.appItemPK = appItemPK;
}
} 


@Embeddable
public class AppItemPK implements Serializable {

private static final long serialVersionUID = 1L;
@Column(name = "app_id")
private Long appId;
@Column(name = "item_id")
private Long itemId;

public Long getAppId() {
    return appId;
}

public void setAppId(Long appId) {
    this.appId = appId;
}

public Long getItemId() {
    return itemId;
}

public void setItemId(Long itemId) {
    this.itemId = itemId;
}

public boolean equals(Object obj) {
    if (obj instanceof AppItemPK) {
        AppItemPK appItemPK = (AppItemPK) obj;
        if (appItemPK.getItemId().equals(this.itemId)
                && appItemPK.getAppId().equals(
                        this.appId)) {
            return true;
        }
    }
    return false;
}

public int hashCode() {
    return this.itemId.hashCode() + this.applicationId.hashCode();
}

}

使用下面的代码将记录插入到 app_item 表中

@Transactional(readOnly = false)
public boolean saveItemSelection(PageHeaderViewData pageHeaderViewData, Map<Long, Boolean> selectedItems,String savedItems){
    long millisSaveStart = Calendar.getInstance().getTimeInMillis();
    log.debug("Inside saveItemSelection appId:"+pageHeaderData.getAppId());
    boolean saveStatus = false;     
    List<AppItem> appItemInsList = new ArrayList<SavedApplicationItem>();

    if (pageHeaderData.getAppId() != null) {

         for (Entry<Long, Boolean> idEntry : selectedItems.entrySet() ) {
             if (idEntry!= null){                    

                     if (idEntry.getValue() && !savedItems.contains(idEntry.getKey().toString())){                           
                         //log.debug("Inside saveItemSelection SAVED itemId:"+idEntry.getKey()+" , Value:"+idEntry.getValue());          
                         AppItem appItem = new AppItem();
                         AppItemPK appItemPK = new AppItemPK();
                         appItemPK.setAppId(pageHeaderData.getAppId());
                         appItemPK.setItemId(idEntry.getKey());
                         appItem.setAppItemPK(appItemPK);
                         appItem.setUpdateInd(ToggleEnum.Y);                        
                         appItemInsList.add(appItem);
                         //appItemRepository.saveAndFlush(appItem);                          
                     }
                 }
             }


         } 


         if (appItemInsList.size() != 0){
             long millisJPASaveStart = Calendar.getInstance().getTimeInMillis();
             appItemRepository.save(appItemInsList);    
             long millisJPASaveEnd = Calendar.getInstance().getTimeInMillis();
             log.debug("JPA save time:"+(millisJPASaveEnd-millisJPASaveStart));
         }

         saveStatus = true;

         long millisSaveEnd = Calendar.getInstance().getTimeInMillis();
         log.debug("Total save time:"+(millisSaveEnd-millisSaveStart));
    }

    return saveStatus;

}//end of saveItemSelection

插入 5000 条记录需要 13826 毫秒。

有人可以告诉我如何提高上述 JPA 代码的性能吗？我们正在使用 hibernate 来实现 jpa。

Answer 1

要提高插入性能，您应该使用自定义代码实现批量插入。下面的方法将确保将批处理刷新到数据库中。根据您的性能测试调整批量大小。通常 50 是一个很好的起点。

@Transactional
public void bulkPersist(List<Entity> entities) {
  int i = 0;
  for (Entity entity : entities) {
    em.persist(entity);
    i++;

    if (i % batchSize == 0) {
      flush();
      clear();
    }
  }
}

以上更改将创建多个插入语句。您可以通过设置 hibernate 配置来进一步优化插入查询。

<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates">true</prop> 

提示:启用调试日志以查看每批生成 1 个插入查询。